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Readers are asked to note that, from now onwards, as part of a general reform 
of the information and documentation services relating to the educational and 
cultural work of the Council of Europe, this Bulletin will no longer contain a 
First Part summarising the results of European meetings held imder the 
auspices of the Council for Cultural Co-operation. Sudi information will be 
inserted, in an abbreviated form, in the review '"Education and Culture*" whidi 
is already received by most readers of the Bulletin. 

Two further dianges should be noted. First, the Bulletin will be published in 
three, four or five issues per annum, according to the nature and the volume 
of the material to be presented. Second, the scientific or policy papers resulting 
from European conferences, colloquies or symposia, whidi henceforth take up 
the whole of the Bulletin, will cover the problems of cultural development as 
well as educational themes. 

Specialists who are particularly interested in national educational reforms in 
Europe are reminded that abstracts describing such reforms continue to be 
published in the "Newsletter" of the Documentation Centre for Education in 
Europe. 




THE EVALUATION OF SCHOOL REFORM 
PILOT PROJECTS 



Educational Research Symposium, Rheinhardswaldschule, 14th— 18th October 1974 



The Educational Research Symposium on the Evaluation of School Reform 
Pilot Projects was the sixth in a series of symposia sponsored by the 
Educational Researdi Committee, The Symposium was organised by ,the 
German authorities under the auspices of the Council of Europe; and was held 
at Rheinhardswaldschule near Kassel from 14th to 18th October 1974. The 
Chairman was Professor W, Mitter, Director of the German Institute for 
International Educational Research, Frankfurt/Main. Delegates from nineteen 
member States — researchers, officials -from education ministries and represent 
tatives of school administrations — participated in the Symposium. 

The objective of the Symposium was to compare methods and results of the 
evaluation of school reform pilot projects and to investigate the scope for 
European co-operation in this field. The evaluation of reform has become one 
of the most vital issues along the borderline between policy and research. The 
rather naive judgments on innovation which are, in the battlefield of ideologies, 
based on an act of faith, are more and more being superseded by scientific 
methods of evaluation of diange. It was acknowledged, however, that differing 
objectives in school reform made the universal application of specific evaluation 
methods doubtful and that evaluation by researdiers could also be abused for 
political ends. 

The Symposium dealt in particular with the general trend towards schooU 
based, de-centralised innovation. The centre-periphery model in which reform 
is conceived at the Ministry and imposed hierarchically on the schools and 
their population has in many countries not been able to live up to the expecta-- 
tions which backed this model. A new model, the periphery-centre^periphery 
model, is therefore developing in a number of countries. At present^ innovation 
often begins at the grassroots and takes the shape of pilot projects in which 
teadters and administrators, parents and pupils, and even the , whole local 
community, take an active part. Such innovation is then reported to the 
Ministry by the Inspectorate, by written reports or by demonstration of its 
results. It finally spreads from the centre back to the periphery in a general 
dissemination process whiih may be strongly reinforced if the evaluation can 
claim to be research-based. 

Thus the Symposium provided a useful opportunity for comparing develop^ 
ments, results and methods in the various countries. General agreement was 
reached that the most useful evaluation projects arc those in which the 
research teams are not neutral observers applying tests at periodic intervals 
from the **outside*', but those projects where the researchers are active partici^ 
pants, together with teachers, administrators and parents, from the very 
beginning of the evaluation to its end. 

All lectures, the discussion group reports and the background documents 
dealing with national evaluation projects are given below, A German version 
of the Symposium papers will shortly appear in a special issue of Bildung und 
Erziehung. 
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. LECTURES 

Comparative aspects of evaluation of school 
reform pilot projects 

by Professor W. MITTER, Director, 

Deutsdies Institut fur Internationale Padagogisdie 

Forsdiung, Frankfurt/Main. 



Introduction 

Sdiool reform pilot projects are today being carried 
out and evaluated in all industrial countries. Thus 
the CoiUicil for Cultural Co-operation of the Coun- 
cil of Europe deserves thanks for making this 
symposium possible, affording educational scien- 
tists and representatives of sdiool administrations 
from the member countries of the Council of 
Europe the opportunity to discuss the evaluation 
of sdiool reform pilot projects from the point of 
view of general and specialised theoretical researdi, 
their role in the formulation of educational and 
social policy, and the role of the educational scien- 
tist iii the researdi process, and to compare expe- 
rience regarding the planning, conceptualisation 
and execution of sdiool reform pilot projects in 
general and selected projects in particular. 

The programme of lectures is concentrated on the 
decentralisation of school reforms, the development 
of sdiool-based evaluative projects and illustration 
of the interaction between aspects of organisational 
theory and methodology. The lecture titles them- 
selves, with their high level of abstraction, show 
that the subjects selected do not directly relate to 
national experience. This is deliberate, insofar as 
any tendency to reduce the programme to an addi- 
tive sequence of national reports was to be avoided. 
It is true that an analysis of national peculiarities 
would have given the participants items of valuable 
information, but it would have made the effort to 
define general methodological questions as well as 
the attempt to find and develop comparison crite- 
ria more difficult. For national reports — espe- 
cially when they have the character of a survey — 
tend toward a global view of the subject, even 
when the latter is relatively limited, as in our case 
it is limited to sdiool reform pilot projects. 

At the same time, the decision of the preparatory 
committee to define the lecture subjects in terms 
of theoretical researdi categories does not ignore 
the fact that what is said by the lecturers from 
various countries will reflect experience gained in 
concrete project work carried out under conditions 
set by norms of national ec lational policy and 



scientific tradition. The supra-national significance 
of the entire subject field, as well as the circle of 
participants, will also help in explicitly drawing 
international comparisons in group work and — 
even more important — in guiding the discussions 
implicitly in the sense of a "hidden curriculum". 
This will happen despite the fact that international 
comparison of concrete evaluation projects is the 
specific subject neither of the entire symposium 
nor of individual lectures. The method of beginn- 
ing with the problems expounded in the lectures 
and with the abstraction of experience based on 
analysis of national specifics, and of stimulating 
the participants to make explicit and implicit in- 
ternational comparisons, appears more promising. 
Perhaps the course of this symposium will con- 
firm this assumption — or perhaps it will disprove 
it and thus indicate alternative approadies for fu- 
ture symposia. 

I should like, by way ot introduction to our dis- 
cussion, to draw your attention to some of the 
comparative aspects of the evaluation of school 
reform pilot projects. In the first part of my re- 
marks I would like to discuss arguments in favour 
of legitimising international comparative studies 
within educational research as a whole. The second 
part is devoted to an outline analysis of contro- 
versial ideas put forward in the discussion of eva- 
luation theory in the United States during the 1960s 
and in recent discussion of the function of compari- 
son within evaluative researdi in the Federal Re- 
public of Germany. The insights gained from both 
points will form the basis of an attempt to outline 
— in the conclusion — opportunities for and limita- 
tions on international comparative studies whidi 
could relate specifically to sdiool reform pilot 
projects. 

International comparative studies in educational 
research 

1. What arguments could be given in order to ju- 
stify the necessity of international comparative 
studies in educational researdi? 



1.1 As an instrument of the search for truth and 
hence the humanisation of man's society, science 
fulfils the task of critically testing and clarifying 
naive, pre-scientific descriptions and explanations 
of conditions and events in organic and inorganic 
nature — and thus also in the field of social 
interaction — wherever these are found. Compara- 
tive international studies of situations and events 
in the fiel^ of education conform to this definition 
of the task, tor pre-scientific international compari- 
sons have been made ever since there have been 
rudimentary forms of that social entity, the 
"nation". Just as the rules of education in African 
bush schools were altered as a result of compari- 
son v/ith analogous precepts and rites discovered 
and observed in neighbouring tribes, so did impor- 
tant 19th century representatives of the intellectual 
life of their countries travel abroad to visit schools 
and observe lessons there. From their observations 
and comparisons, they formed recommendations 
aimed at showing up foreign educational practice 
as an example to be followed or rejected. The great 
pedagogical travellers of the 19th century, such as 
Victor Cousin, Matthew Arnold, Horace Mann and 
Leo Tolstoi, are honoured in comparative education 
circles as forerunners and pioneers of the 20th 
century educational studies. 

We realise, of course, that pre-scientific compari- 
sons can lead away from scientific understanding 
of reality and even trigger irrational processes in 
opinion forming. The epistemological function of 
comparative education research is emphasised by 
the presence of such processes. In the Federal 
Republic of Germany, for instance, the term 
"Swedish school reform" has been degraded to a 
provocative slogan as a result of naive comparisons 
by both supporters and opponents of the 
comprehensive "Gesamtschule"; the same thing 
happened to events and measures which became 
known in fragmentary and distortea fashion from 
the American educational system. The fact that 
naive comparison is also capable of producing 
paradoxes is shown by the reception given to 
information about the educational system of the 
German Democratic Republic; the paradox here is 
that approval is based not only on ideological 
commitments to socialist quality (not under dis- 
cussion here), but also on superficial admiration of 
"discipline" and "achievement" which betrays a 
conservative attitude and does not take into 
account the political, social and ideological back- 
ground to individual educational and pedagogical 
measures. 

1.2 Comparative international studies are prompted 
by the logical discovery through research that there 
are phenomena in the field of education which can 



be explained only in terms of national circumstan- 
ces or which are best explained with reference to 
the nation as a social entity. Hence the choice of 
the nation as a unit of study for certain topics of 
educational research. For most of Europe this at 
least seems to provide a working definition, as is 
very well evidenced by the contents of the survey 
on educational research policy in Council of Europe 
member state*;, produced in 1973 by the Documen- 
tation Centre for Education in Europe. While 
contenting ourselves with a working definition, we 
by no means fail to realise that thorough clarifi- 
cation of concepts constantly requires new theore- 
tical efforts. These need to be tested empirically 
as well as badced up by reference to the historical 
development of the concept of the "nation", whose 
ambivalence is reflected in the distinction between 
J. G. Herder's philosophical interpretation in tern-3 
of history and culture (the "nation" as a natural and 
organic linguistic community) and J.J.Rousseau's 
emphasis on political choice (the nation as a 
political community determined by "volonte g4n4- 
rale"). 

Comparative educational research is forced to 
participate in this continuous clarification of con- 
cepts by current events "s well, whether these 
concern the federalisation of educational systems 
on ethnic principles, as in Belgium and Czechoslo- 
vakia, or the new minority problem of migrant 
workers (Gastarbeiter). These are marginal pheno- 
mena in European educational terms, important 
though they may be to the countries concerned. But 
the necessity mentioned above of using the social 
entity "nation" as a frame of reference to explain 
"normal" features of education and educational 
policy can be demonstrated using numerous 
examples. We shall select a few, first of all from 
the area of school structures. 



Structural features of educational systems 

Example 1: As soon as the topical question of 
access to higher education is broached in interna- 
tional terms, educational policy-makers and re- 
searchers are confronted with national regulations 
governing the relationship between school-leaving 
examinations and qualifications for university 
entrance, which raise a host of complex issues in 
each individual country. The organisation of a 
"big lift" for West German students to study at 
American colleges would be thus not exclusively 
a problem of financial policy. 

Example 2: A comparative observation of the 
development of comprehensive sdiools and "Ge- 
samtschulen" in Great Britain and the Federal 
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Republic of Germany must take into account the 
highly developed and socially accepted British 
system of "public" (private) sdiools, which offers 
parents who do not wish to send their children to 
comprehensive schools a real alternative, even 
when there are no state grammar schools available. 
In the Lander of the Federal Repul)lic of Germany 
political decisions to make the integrated Gesamt- 
schule the norm would have much more impact on 
the parents of individual ten or twelve-year-olds. 

Example 3: The United States Supreme Court's 
most recent decision against the "bussing" intro- 
duced in the 1960s and against the related organi- 
sation of suora-local school administrations was 
based specifically on the grounds that local control 
of the sdiools was a central principle of educational 
policy whidi must not be infringed. Participants 
in a symposium on the decentralisation of school 
reforms in Europe might find food for thought in 
this argument, regardless of the intra-national 
significance and effect of this ruling. 
These three examples were chosen to demonstrate 
structural features of educational systems which 
are due to national characteristics, but they may 
also serve to draw attention to curriculum charac- 
teristics. Thus the relationship between the comple- 
tion of secondary education and university entrance 
also depends on differences in curricula at equiva- 
lent levels in different countries; it should also be 
mentioned that the "bussing" decision marked the 
end of experiments in the area of social science 
curricula which had been supported by the State 
for a decade. 

The function of history teaching in the process of 
political socialisation may be taken as a typical 
example of a concentrated study of curriculum 
characteristics; the International Sdiool Book Insti- 
tute in Braunschweig has been engaged in such a 
study for more than two decades. The teaching of 
mathematics should also be analysed in the context 
of a comparative study of national educational 
strategies before interpreting the significance of 
international achievement tests such as the lEA's, 
for the criteria and test procedures used as a basis 
for assessing the effectiveness of mathematics 
teaching are fundamentally open to question. Were 
adiievements in mathematics — we would have to 
ask — tested primarily from the point of view of 
applicability to scientific and tedinological discipli- 
nes, or was sufficient consideration given to the 
fact that a national educational system might 
regard it as the main function of mathematics 
instruction to develop young people's ability to 
think logically and thus afford them a basis for 
philosophical understanding of themselves and of 
the world? International achievement comparisons 
are prone to be misinterpreted and reduced to 



mere pegs for ideological arguments if insufficient 
allowance is made for national background 
variables. 

Lastly, situations and events worthy of study in 
national educational systems can be affected when 
nations are split or absorbed into new social 
entities as a result of political changes. In the field 
of education, traditional structures, contents and 
attitudes relating to the "old" nation outlive such 
changes, which can lead in comparative internatio- 
nal studies to discoveries as theoretically intriguing 
as they are politically significant. One example of 
this is the comparative study of present-day school 
development in the states which emerged from the 
Austro-Hungarian monarchy, although here the 
question of whether the term "nation" can be 
applied to nationality states is a matter for further 
inquiry and reflection. 

1.3 In the age of international communication, the 
development of clear terminology is of the greatest 
importance, as may be seen for instance from the 
compilation of multi-language thesauri. As regards 
the contribution comparative international studies 
can make in this respect, two types of terminology 
deserve special mention. 

Where research theory and method are concerned 
there is a supra-national language commonly used 
in the empirical social sciences. As regards the 
subject of this symposium, for example,. I am 
thinking of terms used to define research models, 
research types, investigative procedures and data- 
gathering methods. Linguistically of course, this 
supra-national language is derived primarily from 
one national ie, English, source. This concentration, 
makes it easier to acquire and use sociological 
vocabulary, not only for native speakers of English, 
but also for "foreigners", although misunderstand- 
ings may occur owing to the use of English terms 
in other languages without due regard for semantic 
factors. Accordingly, in formulating the overall 
subject of this symposium, it had to be taken into 
account that the German term "wissenschaftliche 
Begleitung" has no literal counterpart in English 
or French and that the English term "evaluation" 
is not covered by the German borrowed term 
"Evaluation". The danger of misinterpretation is 
specially large when theoretical ideas and findings 
are communicated to school administrators and 
teachers concerned with putting them into practice. 
The involvement of "practitioners" in actual re- 
search, as in action research, increases this danger 
but could, due to the permanent necessity of clari- 
fying ideas in practical everyday work, help to 
reduce the number of "fashionable" terms and thus 
— in the field which concerns us here — to develop 
meaningful supra-national scientific language. 
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Mudi more difficult than fixing research theory 
and method terminology is the attainment of 
linguistic clarity in the description of national 
educational systems within whidi — in our case — 
the sdiool reform pilot proje/Jts and evaluation 
projects to be analysed are carried out. This applies 
not only to the structural framework, but also to 
curriculum development and to socio-pedagogical 
factors in the broadest sense of the word. Major 
problems arise here not only in the dissemination 
of researdi findings, but in the course of researdi 
itself, at the definition stage. The analysis of 
general aims and the definition of operational 
objectives, to take one important example, require 
a consensus on the terms to be used before content 
factors are worked out unless the discussion is to 
be reduced to mere controversy or refuge is to be 
sought in terms whidi are only apparently 
unambiguous. An example is the Frendi term 
''d^mocratisation", whidi refers to equal opportu- 
nity in sdiool attendance and achievement, and 
does not mean the same thing as the West German 
"Demokratisierung", whidi expressly covers the 
extension of pupils', teadiers', and parents' partici- 
pation in decision-making. Misunderstandings are 
particularly frequent in the field of political edu- 
cation, where national usage is overlaid with 
non-specific ideologically slanted terminology; the 
confusion is particularly starkly revealed in the 
unthinking use of sudi basic concepts as "demo- 
cracy" and "peace". 

The classification of sdiool structures is compli- 
cated by the fact that the terminology tends to 
reflect historical tradition rather than present-day 
functions. Traditional terms remain in use and are 
even — as in the case of the "Gymnasium" in 

, Czechoslovakia — re-introduced, even where the 
t^pes of school to whidi they refer have changed 
radically in the meantime in both structure and 
curriculum. One example may suffice to illustrate 
the problem: in the present sdiool system of the 
German Democratic Republic the term "Ober- 
sdiule" denotes classes 1 to 12 of general education, 
whidi' as the "Allgemeinbildende polytedinisdie 
Obersdiule" is the general sdiool for classes 1 to 10 
for all sdiooldiildren and has nothing in common 
with the former selective German "Oberschule". 
Problems of understanding whidi arise here simply 
from a "German-with-German" comparison are 
further increased if, for example, the American 
"high sdiool" is translated into German without 

. qualification as "Obersdiule". 



Pragmatic considerations 

1.4 Whereas the arguments advanced so far justify 
the choice of thejnation as a unit of study on 
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theoretical grounds, the remaining two are based 
on pragmatic considerations. The first is that in 
gathering data educational research can most 
readily draw on the data bases prepared by the 
documentation centres of individual countries and 
supra-national institutions — sudi as the Docu- 
mentation Centre for Education in Europe and the 
International Bureau of Education, whidi are 
continuously kept up to date and organised, the 
surra-national centres basing their own classifica- 
tion on national categories. This affords educatio- 
nal researdi access to material which could 
otherwise be obtained only with great difficulty 
and subject to limitations or not at all. Suppliers 
of data are, on the one hand, the central statistical 
offices (with their brandies) whidi compile educa- 
tional and social statistics, and on the other the 
documentation services, whidi systematically collect 
primary verbal sources sudi as laws, regulations, 
statutes, etc. 

The abundance of data thus classified encourages 
comparisons, and the increasing intemationalisation 
of data collection creates new opportunities. How- 
ever, there are dangers for the educational 
scientist who seeks to discover pedagogical and 
anthropological phenomena, as may be illustrated 
by two problems: 

Problem 1: The statistical data bases and verbal 
documentation prepared by state agencies are first 
of all incomplete and — with regard to international 
applicability — are not compiled according to 
standard criteria. Although considerable improve- 
ments and refinements may be expected here in 
the near future, the more central state bodies 
monopolise data collection and storage, the less 
this is true of doubts about the reliability of 
published data. Comparative studies whidi go 
beyond the context of the national state as a self- 
contained unit and select regional phenomena 
such as evaluative researdi projects, for instance, 
as subjects of comparison, would be welcome from 
the point of view of complementary data collection 
and hence for the questioning and correcting of 
official national statistics alone. 

Problem 2: Another problem is that the relatively 
easy access to data tends to make educational 
science dependent on the "pedagogy of the school" 
and reinforces a tendency to over-estimate the 
influence of public and private institutions on the 
young and even on adults. One d6es not have to be a 
"de-sdiooler" to have doubts about sudi tendencies. 
The distinction whidi has become current in the 
empirical social sciences between "international" 
and "inter-cultural" comparison, concentrating 
on comparing socialisation processes in social 
groups, is certainly correct and necessary in cate- 

7 

i) 



gorical terms. It, however, this should lead to the 
creation of distinct types of research pursued 
independently of each other, educational theory 
and practice would both suffer in that those 
involved in the educational process — teachers and 
pupils — would be investigated through two 
isolated comparison procedures: on the one hand 
through de-politicised *'inter-culturar' comparison 
and on the other through an "intemationar* 
comparison in whidi the working concept "nation** 
became a stable term whose use could lead to 
exaggeration of the importance of national back- 
ground variables. 

1.5 The last argument can be stated very briefly: 
comparative international studies are prompted 
and justified by the increasing intemationalisation 
of educdtxonal policy and by the world-wide and 
regional- activities of international and supra- 
national organisations. 

2. We now turn to the second question, the 
problem of the applicahility and usefulness of 
comparison in evaluative researdi. 



Outline analysis of controversial ideas in the eva- 
luation theory 

2.1 Let us begin with opinions on comparison as a 
researdi tedinique that were defended in the 1960s 
in theoretical discussion in the United States on the 
question of curriculum evaluation* The controversy 
was directly triggered by the question of whether 
new curricula should be evaluated by comparison 
with control groups taught according to conven- 
tional curricula. It had been found that efficiency 
tests frequently showed no significant differences. 
The question v/as accordingly: can or should new 
curricula be subjected to evaluative comparison 
with conventional curricula? The two conflicting 
answers to this question were represented in the 
American literature by Lee J. Cronbadi and 
Midiael Scriven. The essential arguments in this 
controversy are outlined below. It is emphasised 
that this brief analysis does no more than describe 
examples, and in particular that the later researdi 
findings of the writers mentioned are left out of 
account. Cronbadi (1963) contests the value of 
comparative evaluation. Firstly, he draws attention 
to its inconclusiveness, as revealed in the non- 
significant differences in average test results, and 
to sources of error attributable to the teadier's 
commitment, whidi can scarcely be measured and 
whidi in turn influences the pupils* motivation 
during the experiment It is never possible to say 
with certainty whidi variables explain the observed 
and tested advantages of a new curriculum. Even 



in the case of extensive, well-plaimed and correctly 
controlled comparative studies therefore, it is 
dangerous to generalise from the results. Secondly, 
Cronbadi raises the question of the often different 
objectives of the currictda whidi are being com- 
pared. Except in the rare cases where one of the 
curricula could be clearly shown to be better, 
comparative interpretation is hampered by the 
difference in objectives. Finally, he raises the 
pragmatic question of the financial outlay in rela«> 
tion to the expected si^uficance of the results; he 
regards this question as important enough for the 
usefulness of a comparative evaluation to ' be 
assessed according to whether the ratio between 
the two quantities is reasonable — whidi, as a rule, 
it is not. 



Naive and scientific comparison 

Midiael Scriven (1967) takes ^he opposite view 
that experiments with new curricula are always 
stimulated and spurred on by competition and that 
curriculum evaluation should therefore never 
exclude comparison; in this connection I would 
like to call to mind the relationship mentioned at 
the outset between naive and scientific comparison. 
Cronbadi's example of the new car model illustra«> 
tes the controversy. Cronbadi*s contention that the 
engineer is interested only in the power and 
reliability of the model he is testing is answered 
by Scriven by saying that the only yardstidc of a 
car's power and reliability and our interest in it is 
our knowledge of what has proved possible within 
a certain price range, a certain amount of room 
and a certain total weight. 

As regards the significance of comparative find- 
ings^ Scriven basically agrees with Cronbadi that 
the instruments used for comparative evaluation 
are not yet sufficiently well calibrated. He conclu«» 
des from this, however, that the financial outlay 
needed to make the tests more precise is justified, 
as is an expansion of comparative groups beyond 
the sizes customary hitherto. 

Scriven toudies upon the question of the applicable 
lity of comparative evaluations to educational 
policy to the extent that, in his opinion, the 
values assigned to the content and objectives of 
curricular innovations must be suitably weighted; 
here even relatively small differences in adiieve* 
ment might prove significant. This applies to the 
evaluation of both new and conventional curricula. 
In his view Cronbadi does not fully appreciate the 
fact that the absence of clear differences f recuently 
provides the very evidence needed to decide in 
favour of developing tried and tested curricula 
rather than introduce promising, but radically new 



models. It might be equally wrong, however, to 
discontinue an experiment at an early stage 
because comparative evaluation in the first trial 
had not revealed significant differences, especially 
as the results would probably not provide any 
information about the possible long-term effects of 
curriculum dianges. 

Scriven's theoretical reflections later taken up by 
Robert E. Stake (1968) and supported by Ridiard 
C Anderson (1969), extend beyonsj* the question of 
instrumental precision and applicability to educa- 
tional theory. Here we are concerned with the 
answer to Cronbach's question about differences 
in objectives as an obstacle to comparative evalua<* 
tion. According to Scriven, in the absence of 
comparison curriculum evaluation could be carried 
out only by means of tests against absolute 
standards. This, he claims, raises the question of 
absolute criteria, especially as experience has shown 
that scales used in non-comparative evaluations 
are usually percentage scales or scales with implicit 
comparisons. Thus terms such as "useful" or 
''valuable contribution*' as value judgements of 
curricula always contain an element whidi indi- 
cates their superiority over other curricula, with 
whidi they are implicitly compared. 

Stake traces the setting of absolute standards to 
personal judgements on the part of the evaluators; 
it seems to me that attention should rather be 
drawn to the influence of economically, politically 
and ideologically determined group interests. With 
his evaluation matrix. Stake has devised an instru- 
ment for the orientation of objectives in terms of 
absolute as well as relative standards; it emphasi- 
ses the value of evaluations whidi include the 
comparison of relative standards in determining 
positions on the scale. 

The relevance of the researdi theory controversy 
to educational decision-making isloudied upon by 
Stake in raising the question of target groiQ>s. He 
argues that the practical teadier who actually has 
to make a dioice of curriculum will tend to find 
Scriven's positive attitude to comparative evalua- 
tion more plausible than the curriculum innovator 
or the educational tedinologist. 

2.2. Most of the argi^ments expounded in this dis- 
cussion of evaluation theory have been repeated 
in recent controversies about the use of compara- 
tive studies in evaluative researdi in the Federal 
Republic of Germany. I shall confine myself here 
to opinions formulated by representatives of sdiool 
adinlnistrations or members of state institutes of 
educational researdi. 

They refer to the recommendation passed by the 
Education Commission of the German Educational 



Council (Bildungskommission ~ Deutsd)er Bil* 
dungsrat) on January 31 1969, on the "Introduction 
of Comprehensive Sdiool Experiments'* whidi also 
emphasises scientific control of the projects and 
the significance of evaluative researdi. 

The Educational Council's recommendation dis- 
tinguishes three types of pilot-project evaluation: 

1. Comparisons of the .system as a whole with the 
traditioiial tripartite sdiool system ("inter- 
system comparisons**), 

2. Comparisons of the structures of the new 
system with its intentions ("system-immanent 
comparisons**), 

3. Comparisons of variants within the comprehend . 
sive sdiool system ("intra-system comparisons**). 

Before going into the various reactions to this 
recommendation, I should like to point out that the 
experimental situation differs from curriculum 
innovation in America by its greater breadth and 
complexity. The- West German comprehensive 
school pilot projects are concerned not only with 
testing new curricula, but also with testing a type 
of sdiool whidi differs so greatly from traditional 
sdiools in the tripartite system with respect to 
both pupil population (social origin, level of 
knowledge, intelligence) and objectives that the 
differences in objectives discussed by American 
educational scientists stand out mudi more sharply. 
In addition, the introduction of the new sdiool 
type necessarily entails testing forms of organisa* 
tion based on external and internal differentiation 
whidi unlike similar arrangements in traditional 
secondary sdiools are central to the system. 

For the purpose of illustrating the main issues, we 
shall take opinions from Baden-Wuerttemberg, 
Bavaria, Hesse and Lower Saxony, concentrating 
on the centres of controversy and beginning with • 
the general remark that all the au^.hors are aware 
of the subtle difficulties with whidi project plan- 
ning has to cope in actual practice. 

Need for system immanent comparisons 

There seems to be general agreement on the need 
for system-immanent comparisons covering paral« 
lei courses in the fame ^ilot sdiooi, they are 
valued as an instrument for improving the results 
of experiments, as well as for correcting partial 
objectives. The authors differ, however, as to the 
possibility o^ intra-system comparisons and especi- 
ally of inter-system comparisons. 

Strong reservations can be found in the documen- 
tary material ("Unterlagen") published by the 
Ministry of Education of Loti^er Saxony. Doubt is 
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cast on the uiefulnets of exact comparative studies 
of the Gesamtschule with the traditional sdiool 
system in general, on the grounds that there is no 
common basis of comparison owing to differing 
objectives and marginal conditions. The reserva* 
.tion about intrasystem comparisons is limited in 
that thematically limited projects are considered 
possible under certain circumstances. But even 
this concession is outweighed by the negative 
overall attitude which classifies comparisons of 
different Gesamtschule projects as ^^methodically 
doubtfur. 

The evaluative research projects Initiated by the 
Hesje Ministry of Education point in the same 
direction, particularly in the reference to a gradual 
shift from efficiency tests and system comparisons 
in favour of system-immanent stydies. The only 
reason given for this is the discrepancy between 
the two school systems* objectives and the means 
of attaining them. 

The Bavarian State Institute of Educational 
Research and Planning holds ti,^ opposite view. 
The evaluation of comprehensive school pilot pro- 
jects is clearly slanted towards efficiency control; 
inter-system comparisons are accepted and control 
schools (from the tripartite system) are included in 
the research schemes. 

The strategy worked out by the Institute of Edu- 
cational Planning and Information in Baden- 
V/utfttembtrg resembles the Bavarian approach in 
that it provides for all three types of comparison. 
However, intra-system comparisons and especially 
inter-system comparisons are made subject to a 
number of criteria and conditions: guarantee of 
random sampling in school selection, great homo- 
geneity of control population, multivariate diarac- 
ter of the project plan, consideration of the 
"conflict-reducing" effect — eg through the parti- 
cipation of psychologists — limitation of problem 
scope in the light of the comparison-hindering 
influence of differepxes in objectives etc. These 
provisos amount to a general reservation which — 
as far as practicability is concerned — comes close 
to the position adopted by Hesse and Lower 
Saxony, though in less drastic form. 

The above remarks have been intentionally con- 
fined to opinions which — although not to be 
regarded as official statements of educational 
strategy — certainly give some clues to official 
policy. The differing interpretations of the German 
Educational Council recommendation reveal funda- 
mental differences in the intermediate-range objec- 
tives of secondary school development, which brings 
us bade to the target-group problem mentioned by 
Bobert E. Stake. In this particular case the fact 



that the research is commissioned by the same 
agency as is intended primarily to benefit from it 
is a major factor in the appreciation and control 
of comparative studies. 

The opinions from Hesse and Lower Saxony are 
based on educational policy strategies which are 
oriented toward introducing the **integrierte Ge* 
samtschule** as the norm and emphasise the dlscre* 
pandes between the objectives of the dedred and 
the traditional school s;nrtem by attributing general 
qualities such as **equal opportunity*" ''social inte- 
gration**, '*individualisation of learning** and **care 
for the socially deprived** ohly to the new system. 
The opinions from Bavaria and Baden-Wuerttem* 
berg, on the other hand, drav attention to strate- 
gies in which the dedsion for or against the 
general introduction of the Gesamtschule remains 
open and the function of ^evaluative research is 
viewed primarily not as optimising school reform 
pilot projects, but as advising educational policy 
bodies which have not yet made up their minds on 
the basic issue. Where the Bavarian position is 
concerned, the openness of the decision «s an 
objective is limited by the research plans to the 
extent that the Gesamtsdiulen are required to 
prove their quality in terms of their superiority 
over traditional secondary schools, whilst its rever- 
se is not expressly provided for. 

Since the types of comparison illustrated by current 
evaluation strategies in the Federal Republic of 
Germany can also be found at the level of inter- 
national comparison, I do not intend to make a 
separate appraisal of the attitudes described. We ^ 
may, however, draw the ^bvioiui conclusion that 
decisions for or against comparison in evaluation 
strategy and the grounds for such decisions appear 
to be influenced by the expectations of the initia- 
tors coinciding with the addressees of evaluative 
research, even more than by the production inten- 
tions of the researdi team. 



Applicability of school reform pilot projects 

3. Our last question concerns the applicability of 
school reform pilot projects for purposes of inter- 
national comparison. Our answer is intended as an 
attempt to put forward general ideas on explaining 
this problem and thus to provide a framework for 
the discussion of concrete research projects. 

3.1 It is difficult to classify comparisons of school 
reform pilot projects in two or more countries into 
one of the three categories described earlier. From 
a formal point of view, the subject matter places 
them in the category of intra-system comparison if 
the projects are carried out at the same or similar 
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levels and in the same or similar types of school. 
Examples might be comparisons of Gesanitsdiulen, 
comprehensive sdiools and colleges d*enseignement 
secondaire. It nevertheless appears justified to 
classify sudi comparisons also as inter-system 
comparisonSt because on the basis of the input* 
output model the input variables, the operational 
variables and, especially, the output variables 
determined by the pilot sdiools' objectives and 
intentions differ so mudi in eadi case that forma* 
.tion of a hypothesis should — in order to be useful 
— proceed from the assumption of independent 
**systems'' as subjects of comparison. This option is 
lent additional legitimation in cases where pilot 
projects within a sdiool type introduced as the 
norm — e. g. in the upper classes of the elementary 
sdiool in Sweden — are to be compared with 
projects whidi are either islands in a sea of sdiools 
of another type — e. g. Gesamtsdiulen in Bavaria — 
or have been initiated in a r^on whidi has a 
mixture of existing types — e.g. comprehensive 
sdiools in Great Britain or Gesamtschulen in Hesse. 
In order not to limit the discussion to Gesamtsdiule 
projects, the pre-sdiool field may be mentioned as 
a second example. In this case pilot projects may 
be treated in terms of different "systems**, if there 
are substantial differences in the proportions of 
four or five-year-olds in pre-sdioo) education in 
relation to the total populations in that age group. 

The most obvious subjects of comparison are indi- 
vidual sdiool reform pilot projects, although the — 
admittedly complicated — comparison of project 
planning and the progress of experiments within 
comparable regions should also be considered. 

3.2 Inter-system comparisons on an international 
scale belong in the problem approadi category as 
developed by comparative educational science, 
whidi has displaced the formerly dominant total 
analysis owing to the growing realisation that 
demands made by this form of study are usually 
too wide and cannot be satisfied by empirical 
researdi. In terms of our subject, this means that 
the implementation of sdiool reform pilot projects 
can be defined as a "problem** and hence as a basis 
for comparison. The aim of the comparison could 
then be to establish whether and to what extent 
the educational policy strategies of the "home 
countries** are reflected in the pilot projects under 
comparison — in the sense of functional equiva- 
lents. A prerequisite for sudi project planning 
would be previous system-imminent comparisons 
within the sdiool systems to be compared. 

The purpose of inter-system comparisons on an 
international scale may generally be explained by 
reference to the arguments advanced at the outset 
to justify comparative international studies, in 
particular with the qualification of the central 
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argument to the effect that the existence of the 
nation as a social entity is most important for 
the preparation of a catalogue of variables — 
through the inclusion of appropriate badcground 
variables on the output side. This produces effects, 
firstly on reservations about hasty and xmfiltered 
adaptations of foreign reform projects and, second- 
ly, on the more precise definition of intranational 
evaluation projects through greater regard for na- 
tional educational traditions whose suppression is 
always reflected post facto in the failure of st\di 
projects. 

3.3 International inter-system comparisons confront 
a researdi team — if only because of the dif- 
ficulties in getting started and in apportioning 
costs — with the necessity of planning their pro- 
jects particularly carefully. This applies first of 
all to investigation of the quality criteria whidi 
have to be observed in all empirical studies. Inter- 
national studies can thus contribute to improving 
methods and instruments whidi are important to 
evaluative research in general. A quality criterion 
whidi demands additional attention and whidi 
would have to be included in a "methodology of 
comparison of sdiool reform pilot projects** would 
be terminological uniformity, whidi would require 
fixation of a standardised language for the coun- 
tries to be compared. 

3.4 The successful implementation of comparative 
international studies would seem likely to be a 
way of reducing apodictic judgements on the du- 
bious value of inter-system and intra-system com- 
parison as researdi methods. This assumption re- 
fers to the question of differing objectives whidi, 
as an obstacle to the establishment of a basis for 
comparison has been a cause of concern to Ameri- 
can curriculum evaluators, as well as to project 
planners in the Lander of the Federal Republic 
of Germany. However, even when the reservations 
resulting from a consideration of this problem 
have been taken fully into account, the following 
arguments speak strongly in favour of compari- 
sons in the evaluation of sdiool reform pilot pro- 
jects. 

First of all, I would draw attention to the signi- 
ficance — mentioned by Scriven and Stake — of 
the comparison of standards as socio-political and 
philosophical aims whidi determine the operational 
objectives of the curricula under comparison. 
Since in pluralistic societies educational policy- 
makers are confronted with the relationship be- 
tween absolute and relative standards in all their 
decisions, it is important for educational researdi 
not to dodge this issue by alleging, insoluble me- 
thodological difficulties and relying on naive com- 
parison. 
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Secondly, the argument that the objectives of dif- 
ferently structured sdiool systems cannot be com- 
pared ignore the existence of similar, if not iden- 
tical, partial objectives whidi are worthy of com- 
parative analysis, not only from the point of view 
of researdi theory but also from that of educatio- 
nal and financial policy. This statement could be 
verified for the acquisition of cultural tediniques 
as well as for the learning of foreign languages or 
the investigation of topics in the natural sciences. 
It must also be remembered that the identity or 
difference of partial objectives may conflict with 
differences in the field of general educational aims. 
For instance, the debate on the modernisation of 
mathematics teaching, which is a possible subject 
for international pilot project comparison, runs 
largely parallel to general developments in the pri- 
mary and secondary school field. 

Finally, international comparisons can promote 
objectivity in the standards controversy, in that 
detadied contemplation of educational systems in 
whose reform the educational policy-maker or 
researdier is not directly involved may help bring 
about recognition and acceptance of differently 
formulated standards as variations on the same 
absolute principles. 

3.5 This detadiment could also aid in clarifying 
the political frame of reference in whidi not only 
practical researdi decisions are made, but also re- 
search intentions are theoretically formed. In all 
open societies, it draws the attention to contro- 
versial educational policy strategies, and the edu- 
cational scientist who undertakes commissioned 
research — in our case the evaluation of school 
reform pilot projects — accepts specific conditional 
factors. In the countries where wc v,ort, these are 
legitimate phenomena of educational reality. Thus 
there is no reason to suppress them from our con- 
sciousness, because it is the identification of them 
that makes it possible to subject project plans and 
researdi findings to critical scrutiny. 
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Decentralised curriculum development in the 
form of action research 



Introduction 

In my lecture, I should like to discuss some pro- 
blems of a relatively new type of sdiool innovation 
researdi; it is a new type, only the very beginnings 
of whidi have been realised in the Federal Repu- 
blic of Germany and — as far as I know — in other 
European countries. Thus it remains primarily a 
programiTie up to the present. I refer to this new re- 
seardi as ^Decentralised Curriculum Development 
(or sdiool-based Curriculum-development) in the 
Form of Action Researdi**. At present, this cannot 
be the presentation of a completely planned, de- 
tailed model; rather, I can only sketdi a few basic 
problems, without any claim to completeness. 

The basis of my remarks are on the one hand, cor- 
responding scientific publications. On the other, I 
refer to perceptions and experience from two prac- 
tical curriculum-development projects in which I 
have participated or am participating; 

a) The first was a fairly extensive attempt to re- 
vise past curricula for sdiools of general educa- 
tion in the land of Hesse, a project which ran 
from 1969 to 1971 and whidi was unfortunately 
interrupted by the Hessian Ministry of Culture 
at a time when we were just going to take the 
step from basic preliminary reflections to syste- 
matic combination of supraregional curriculum 
planning with decentralised curriculum deve- 
lopment. (^) 



(1) Cf. Klafkl, W., Ungelbadi K. Ch. and Nidclas, H. W. 
^Publishers); Probleme der Curriculumentwicklung, 
2nd edition, Frankfurt am Main, 1972. Bedcer, tt, 
Bonn, P. and Groddedc, N.: "Demokratisierung als 
Ideologie. Anmerkungen zur Curriculumentwidclung 
in Hessen**. In: Betrifft: Erziehung, 1972, number 8, 
pages 19-29. "Gravierende kulturpolitisdie Ffehlent- 
sdieidung", interview with Moos, G. and Klafki, W., 
In: Betrifft: Erziehung, 1972, nuiiiber 9, pages 25-28. 



by Professor W. KLAFKI, 
Marburg University. 



b) The second practical source of my considera- 
tions are experiences from two projects, one of 
whidi being carried out by Marburg University 
in cooperation with groups of teadiers at the 
primary school and directed by myself. This is 
a contribution to curriculum reform in the 
Grundschule in the form of action researdi O 
The second is an action researdi-project too, 
whidi so far was carried out by a combined 
working group "Curriculum/Social sciences" at 
the Bildungstedinologisdies Zentrum Wiesbaden 
and to whidi I was attadied as a scientific ad- 
visor. (*) 

In the first section, I will begin with some remarks 
on the main terms whidi I will be using. 

In the new researdi type "decentralised curriculum 
development in the form of action researdi" — 
and here I come to the second point of my paper — 
two activities are combined, although they do not 



(2) Cf. Klafki, W.: "Handlungsforsdiung in Sdiulfeld". 
In: Zeitschnft fur Pddagogik, 1973, pages 487-516, 
especially pages 492 ff. 

(3) Cf. the articles in Beitr&ge zur Bildungstechnologie 
(the journal published by the "Bildungstedinologi- 
sdies Zentrum") in particular No. 3/1972 and No. 2/ 
1973. Instead of the series "Project I: Untersudiung 
und Entwidclung einer lehrer-bezogenen Strategie 
fUr Curdculuminnovation und emanzipatorisdien 
Medieneinsatz", hectographic copies of whidi have 
also been published by the Curriculum/Social 
Science Working Group, I refer to the synopsis of 
the project to be published shortly: Heinze, T., Miil- 
ler, K, Stidcelmann B. and Zinnedcer T.: Hand-- 
lungsforschung im pddagogisdien Feld, Munidi, 1975. 
Under the Coalition Agreement of the new Provin- 
cial Government of Hesse (SPD/FDP) signed in De- 
cember 1974 the "Bildungstedinologisdies Zentrum" 
is to be disbanded. It can therefore be taken for 
granted that the sdieme of the above mentioned 
group will be either discontinued or else continued 
in a very abridged form. 
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necessarily belong together. On the one hand, de- 
centralised curriculum development exists without 
any connection *to researdi, especially action re- 
seardi (*), and on the other, action researdi is not, 
as you know, necessarily related to curriculum 
development. (*) However, I think it will become 
clear in the course of my remarks why action re- 
seardi must be combined with curriculum devel- 
opment as we understand its aims and tasks. 

In what sense will I be using the terms "curricu- 
lum" and "curriculum development"? I use them 
in a wide sense. 

The terms "curriculum" and "curriculum develop- 
ment" refer then to decisions and reasons for deci- 
sions on three levels, i.e.: 

Decisions and reasons for decisions as to 

• general aims and special learning objectives 

• subjects and contents of classroom instruction 
oriented to these aims and objectives 

• the organisational forms, methods and media of 
classroom instruction oriented to the aims and 
objectives and appropriate to the subjects and 
contents. 

The term "decentralised" implies that this kind of 
curriculum development takes place not only in co- 
operation with teachers but also in direct connec- 
tion with their teaching practice, and if possible, 
with the co-operation of pupils and parents. The 
next step is to explain the term "action research" 
as I use it here; in this connection, I always mean: 
ccfucational action research. This term can be tem- 
porarily defined by three general characteristics; 
some further characteristics will be mentioned 
later: 

a) In its interest in knowledge and thus in its ques- 
tions, action research is related from the very 
outset to teaching practice, and this practice is 
understood as a specific form of social practice. 
Action research attempts to solve practical 
teadiing problems. Thus it does not consider it- 
self purely theoretical and neutral research. 

b) Action research is carried out in direct combi- 
nation with the practical problem-solving at- 
tempts which it makes. As research it inter- 
venes directly into practice, not waiting until 
the research process is completed; therefore, it 
must remain open to retroactive effects from 
this practice, which it itself helps to influence, 

(4) Cf . for example Burst, R. et al: Weinheimer Gesamt- 
sdiul'Cunicula, Heidelberg, 1971. 

(5) Cf. for example the majority of contributions to 
AktionsfoTsdiung, published by Haag, F., KrUger, H., 
Schwarzel W. and Wildt, J., Munich, 1972. 



on the questions and research methods — • 
during the researdi process itself, not waiting 
until the final evaluative stage with respect to 
future research. 

c) Action research must try consciously and pur- 
posely to eliminate the division between resear- 
chers and those involved in school practice. Its 
object is to put both groups in the innovation 
and research process into a relationship of direct 
co-operation and mutual influences. (*) 

Basic problems of decentralised curriculum devel- 
opment in the form of action researdi 

After this introduction, which served mainly to 
clarify terms, I now come in the main section of 
my lecture to the discussion of some basic problems 
of decentralised curriculum development in the 
form of action research. Some of these problems 
have already been briefly mentioned in the intro- 
duction. 

As far as I know, no historical presentation of the 
development of the concept of decentralised curri- 
culum development combined with action research 
exists, and I am not in a position to offer such a 
history here. One of the precursors of this new 
type of research is, as I understand it, Dewey's 
school reform work before and at the turn of the 
century. C) A further important step in America 
was taken by Hilda Taba's curriculum work and 
her beginnings of a theory of curriculum develop- 
ment. O But of course, these are only very sporadic 
remarks. 

In Europe, the first beginnings of a decentralised 
curriculum development, only parts of which, how- 
ever, were and are accompanied by action research, 
have been observed primarily since the late 1960's 
probably most clearly in England and the Federal 
Republic, but also in l^weden and Switzerland. (•) 

At least two basic motives are recognisable which 
have led to this development: One motive refers 



(6) See the article mentioned in footnote (2). 

(7) Cf. Bohnsacic, F.: Erziehung zur Demokratie. John 
Deweys Pddagogik und ihre Bedeutung fUr die Re 
form unserer Sdiule. Unpublished thesis presented 
for the lecturer's qualifying examination, Marburg, 
1974. A more extensive version is due to be pub- 
lished in book form by Otto Maier-Verlag, Ravens- 
butg, 1976. 

(8) Taba, H.: Curriculum Development. Theory and 
Practice, 3rd edition, New York, Chicago, San Fran- 
cisco, Atlanta 1970. 

(9) Cf. Gerbaulet S. et al: Sdiulnahe Curriculument- 
wicklung, Stuttgart 1972. — Aregger, K.: Interaktion 
im lehrerzentrierten Curriculumprozep, Weinheim/ 
Basel 1973. 
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to the failure of centrally developed, teadier-proof 
large curricula in the US. This failure was so 
alarming to curriculum theory and practice in 
England and the Federal Republic because many 
curriculum theories and some practical curriculum 
projects in these countries had tried to orient 
themselves to a large extent to American examples. 
The second motive is, I believe, more significant; 
it is a definitely political motive. 

Everyone demanding consistent democratisation of 
sbciety as a whole and the sdiool system in parti- 
cular, i.e., self-determination and co-determination, 
self-responsibility and co-responsibility, must also 
relate this principle to curriculum development 
and curriculum researdi. If curricula for the edu- 
cation of young people in a democratic society are 
supposed to be consistently oriented to the prin- 
ciple of democracy, that is if they are supposed to 
help young people acquire the ability to apply self- 
determination and co-determinatioh, and if teadiers 
are supposed to be in a position to teach in the 
sense of such curricula, then it would be unthink- 
able for these teadiers not to be involved in the 
development of these curricula and the decisions 
put into them. Teadiers who are not given the op- 
portunity to help decide on aims, objectives, con- 
tents, organisational forms, methods and media of 
their classroom instruction cannot train young 
people to apply self-determination and co-determi- 
nation! 

This means nothing less than that curriculum de- 
velopment cannot be carried out according to a tech- 
nocratic model "from top to bottom", not accord- 
ing to the model called engineering or research- 
development-dissemination model. It is the model 
which has been practiced since the 1950's and par- 
ticularly since the early 1960's in the US and occa- 
sionally in Sweden. ('°) Most of the forms in which 
curriculum development or guideline development 
have been carried out up to now in the Lander 
of the Federal Republic cannot be termed realisa- 
tions of this research-development-dissemination 
model, because they usually lade the scientific 
aspect, that is the research aspect. But with regard 
to the diaracteristic "from top to botton", they are 
still comparable to the RDD model. Perhaps we 
could speak of a model abridged by the research 
element, that is of a development-dissemination 
model. Here I cannot go into a description of the 
process of criticism of the engineering model for 
curriculum development and the counter-develop- 
ment of a non-technocratic model, a conception of 
curriculum development closely related to practice. 
I can only indicate a few names and institutions 



(10) Cf. GSerbaulet. S. et al: Sdiulnahe CurriculumenU 
widclung, Stattgart, 1972, in particular pages 50 ff. 
and64ff. 




which have played a part in it. In addition, I must 
emphasize that the stress on the political aspect of 
this countermovement varies widely from one au- 
thor or institution to the next. 

For America, we might mention beginnings in the 
sense of the so-called problem-solving model prac- 
ticed, for example, at the Humanistic Learning 
Center in Albany, New York. In addition, there 
are in America — and especially in the United 
Kingdom — at least some projects on decentralised, 
open curriculum development within the Teadier's 
Centers Movement. ('0 We could also mention the 
English Nuffield Junior Science project and 
the approadies of Lawrence Stenhouse (^^), which 
indude some aspeci^s of decentralised curriculum 
development. 

In Sweden, the work within some of the so-called 
"pedagogical development blodcs** formed in Mal- 
mo, since 1964, for example, is also being done 
along these lines. ('^) 

Perhaps it is only due to a ladc of Information that I 
have the impression that theoretical interest in pro- 
blems of decentralised curriculum development 
and open curriculum appears farthest advanced in 
some groups and publications in the Federal Re- 
public of Germany, partly because we have al- 
ready been able to use international experience. 
First of all I should like to mention the excellent 
thesis prepared by a project group whidi was sti- 
mulated by the Founders' Association of German 
Science in 1972 under the title of "Sdiulnahe Cur- 
riculumentwidciung** (Sdiool-based Curriculum De- 
velopment). ('*) This thesis contains a very detailed 
proposal on the formation of regional pedagogical 
centers whidi should stimulate, support, co-ordi- 
nate and evaluate curriculum-development pro- 
jects carried out by teadiers in co-operation with 
research groups. Many suggestions in this thesis 
have been included in a detailed experts' opinion 
by the Education Commission of the German Edu- 
cational Council under the title of "Promotion of 



(11) Gerbaulet, S. et al: op. cit. pages 176 ff. 

(12) Cf. NuSSield Junior Science: Teadiers Guide, 1. and 
2. edition, London/Glasgow, 1968. — Nuffield Ju^ 
nior Science: Mammals in the classroom, London/ 
Glasgow, 1967. — Jung, W.: "Das Nuffield Junior 
Science Project". In: Die Grundsdiule, 1968, No. 3, 
pages 45 ff. — Lucas, M.: "Das Nuffield Junior 
Science Programm**. In: Neue Wege der Schulbio- 
logie in englischen Unterrichtsprojekten, published 
by R. Pfeiffer, Berlin, 1970. — Klewitz, B. and 
Mitzkat, H.: "Nuffield Junior Science Project — 
Didaktisdie Frinzipien und Beispiele**. In; Die 
Grundsdiule, 1973, No. 3. 

(13) Stenhouse, L.: "Curriculumentwidclung als Experi- 
ment". In: ZeiUdiHjt jUr PUdagogik, 1973, S.447ff. 

(14) Gerbaulet, S. et al: op. cit., pages 71 ff. 

(15) Cf. footnote (8); Gerbaulet, S. et al. 
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Practice-Related Curriculum Development", pub- 
lished in 1974. (") Since 1972, the "Curriculum 
Working Group" founded by sdiolarship receivers 
of the Volkswagen Foundation, which publishes a 
magazine, "Subject: Curriculum", devoted parti- 
cularly to the problem of open curricula and show- 
ing mudi in common with this new type of decen- 
tralised curriculum development. ('^) 

In practice^ decentralised curriculum development 
work in the Federal Republic has been mainly done 
in several Gesamtsdiulen, but only to a very limited 
extent with the support of research groups. The 
laboratory sdiool and the Oberstufenkolleg at Bie- 
lefeld University, for whidi Hartmut von Hentig 
gave the decisive impulse for the set-up and de- 
velopment, are two of the few thorough, Jong- 
range projects on decentralised curriculum devel- 
opment in the Federal Republic. 

The first practical models, whidi, although they do 
not fully correspond to the suggestions contained 
in the aforementioned thesis and the opinion of the 
Educational Council, are nevertheless at least 
closely related to these. They are two projects called 
"Pilot Projects on Regional Advanced Education 
for Teachers" whidi have been running in Hesse 
since 1972. These projects concentrate mainly on 
formulating and testing the new Hessian Guide- 
lines for Grundsdiulen and Secondary Level I. 
Here teadier groups cooperating on a voluntary 
basis are supported by small groups of scientists. 
The project group performing this experiment for 
the Ministry of Culture publishes a very interest- 
ing, practice-related magazine called "Curriculum 
Konkret" ("Concrete Curriculum"). H 

Finally, ! should like to name the Grundsdiule- 
project at Marburg University, already mentioned 
earlier, which I direct and which we regard ex- 
pressly as a project on decentralised curriculum 
work in the form of action research, moreover the 
project at the Bildungstedmologisches Zentrum, 
Wiesbaden, also mentioned above. 

Systematic aspects 

After this brief outline dealing .with the beginning 



(16) Deutsdier Bildungsrat, Empfehlungen der Bildungs- 
kommission: Zur Forderung schulnaher Curricu" 
lumentwidclunot Stuttgart, 1974. 

(17) Thema Curriculum, published by the Arbeitskreis 
Curriculum, since 1972» Verlag I^thar Rotsdi, Be- 
benhausen. 

(18) Cf. footnote (3). 

(19) Hentig, H. von: Das Bielefelder Oberstufenkolleg, 
Stuttgart, 1971. By the sanje author: Die Bielefelder 
Laborschule, Stuttgart, 1971. 

(20) Curriculum Konkret Dokumentation eines ModeH- 
versudis, published by the Projektgruppe Regiona- 
le Lehrerfortbildung, since 1973. 
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of decentralised curriculum development and the 
related action research, I shall now turn to the 
systematic aspects of the question. 

All theoretical and practical beginnings whidi I 
have already mentioned include more or less cri- 
ticism of the model of centralised and closed curri- 
cula whidi have been or will be developed accord- 
ing to the engineering model. From this criticism 
of the researdi-development-dissemination model, 
whidi is in contradiction with the democratisation 
principle, we can draw two positive Consequences: 

a) Those active in sdiool practice, particularly tea- 
diers, must be able to participate actively and 
decisively in the process of curriculum develop- 
ment. Increasingly, pupils and parents should 
also be included in this development; however, 
one must not ignore the difficulties presented 
by just this second demand. 

We cannot, however, take for granted that tea- 
diers, pupils and parents possess from the very 
outset the competence, i. e., the perception, 
knowledge and abilities, necessary to take cri- 
tical and productive part in curriculum devel- 
opment work. Anyone who proceeds from this 
false assumption will necessarily produce dis- 
appointment on the part of all concerned. The 
task is rather as follows: the curriculum devel- 
opment process must be considered at the same 
time as a learning process for all those involv- 
ed. This applies not only to pupils and parents, 
but also to teadiers and — as will be discussed 
later — to curriculum researdiers. Here we are 
mainly interested in the situation of the teadiers. 
Up to now, they have generally been able to 
acquire the qualifications for co-operation in 
curriculum development only to a very limited 
extent during their studies. But even if teadier 
training can confer more favorable starting po- 
sitions in this regard in the future, the main 
requirement will still apply that the process of 
work on the development of curricula or curri- 
culum parts (teadiing units or sequences) must 
be organised as a learning process for all con- 
cerned. 

b) A second, positive consequence is the following: 
decentralised curriculum work must aim. for 
open and variable curricula and curriculum 
parts (^M* It would be a contradiction in itself 



(21) Briigelmann, H.: "Offene Curricula". In: Zeitschrift 
fiir P&dagogik, 1972, pages 93 ff. — By the same 
author: "Lemziele im offenen Curriculum". In: 
Thema Curriculum, 1972, No. 2, pages 16 ff. — Mo- 
ser, H.: Handlungsorientierte Curriculumforschungt 
Weinheim, 1974. — Deutsdier Bildungsrat: Empfeh- 
lungen der Bildungskommission: Zur F6rderung 
sdiulnaher Curriculumentwicklung, Stuttgart, 1974, 
in particular pages 120 ff. 
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for a teadier group to want to produce, in co- 
operation with a group of researdiers of curri- 
culum padcages as a result of their decentralis- 
ed development work, i.e., products to be offered 
to other teadiers and teadier groups simply for 
imcritical acceptance and application, like a 
recipe. On the other hand, we must emphasise 
that the "products" of decentralised curriculum 
development can only be exemples, variable 
models for other teadiers. They should, as exam** 
pies, illustrate how certain objectives and sub<- 
jects can be realised in the classroom under cer- 
tain conditions given by the school, the pupils, 
the teadiers and the concrete situation involved. 
They should stimulate other teachers not to 
sdiematic copying, but to analogous realisation. 
At the end of my remarks, I will return to this 
aspect. 



Dei;eloping instruments and results of research 

First of all, however, let us ask ourselves about 
consequences to be drawn from tnese remarks for 
researdi within the framework of curriculum de- 
velopment work. 

Researdi whidi puts itself in the service of this 
kind of curriculum development is also decisively 
influenced by the democratisation principle just 
emphasised. It cannot regard itself as a neutral, 
exclusively descriptive or analytic discipline. In 
fact, it must be given 'the same aims as curriculum 
development. Its questions muf t be directed toward 
finding conditions, opportunities and difficulties in 
curriculum development as a democratic decision- 
making process. Heseardi must be designed in sudi 
a way that it informs all those affected of the con- 
tents, pre-requisites and consequences of their deci- 
sions. Put more precisely, this means that researdi 
should, more than anything else, be an aid to self- 
enlightenment of those involved. 

Action researdi in this sense is thus a special kind 
of "formative evaluation**. Its results should be in- 
cluded as quidcly as possible in the curriculum- 
development process in progress. I will give you 
an example. Let us assume that a group of resear- 
diers and teadiers arrives in the course of a rather 
general discussion on the objectives of democratic 
teadiing at the conviction that pupils should, 
among other things, obtain insight into the social 
interaction process in whidi they are involved in 
order to be able to act more consciously in sudi 
interactions. Now the group has the idea that one 
important aspect of interactions in sdiool is the 
relationship of the pupils to the teadier. How do 
the pupils regard their teadier? As an authoritarian 
official? As a source of information? As too strict 
or too lenient? Are they afraid of him? What do 
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they expect of him? Whidi of his behavior patterns 
do they reject? When do they react positively to 
his suggestions? Do they see any contradictions 
between the aims and principles whidi he expli- 
citly expounds and his actual behavior? And so on 
and so forth. For setting up a curriculum unit on 
this subject, it is centrally important to possess a 
scientific instrument for finding out the pupils* 
answers to the above questions. For example, a 
pupil questionnaire, an interview guideline, a ca- 
tegory scheme for the evaluation of tape recordings 
of class discussions on these questions ("), perhaps 
even a projective instrument similar to a thematic 
apperception test. The results of sudi on inquiry, 
however, must be evaluated as soon as possible and 
put at the disposal of teadiers and pupils as mate- 
rial for further analysis, discussion and changes in 
their interactions in sdiool. . 

This small example already shows how difficult . 
the task is of developing instruments and results 
of researdi whidi can contribute to the self-en- 
lightenment of those involved. I will return to this 
problem later. 

Developing open and variable curriculum parts 

First of all, however, it is important to eliminate 
a possible misunderstanding with regard to the 
range and efficiency of decentralised curriculum 
development. 

Some publications on decentralised curriculum de- 
velopment give the impression that decentralised 
work alone can solve the curriculum problem; de- 
centralised curriculum development is then repre- 
sented as the only alternative to all forms of cen- 
tralised curriculum work. I consider this a serious 
error of grave consequence. 

Decentralised curriculum development in the form 
of action researdi cannot all alone take over the 
task of developing general curricula for entire sdiool 
systems, school types — all Gesamtschulen or all 
Gymnasien of one Land, for instance — or for 
entire school levels — the so-called orientation 
level for pupils agjed ten to twelve, for instance — 
or curricula for entire fields of classroom instruc- 
tion, sudi as science, or for single courses, sudi as 
history, should these encompass several years. The 
specific task of decentralised curriculum develop- 
ment in combination with action researdi consists, 
in my opinion, of developing open and variable 
curriculum parts, namely teadiing units or sequen- 
ces and strategies for sudi curriculum work. 



(22) Cf. Haller, L and Wolf, H.: ''Selbstreflexion der 
Lemgruppe auf ihr eigenes Verhalten als didakti- 
sche Kategorie der pclitlsdien Bildung", In: Fadci- 
ner, K.: Handbuth des politischen Unterridits, 
Frankfurt am Main, 1972, pages 239-289. 
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Sudi a development of curriculum parts usually 
requires an existing general curriculum framework 
or even better, at least two sudi frameworks: on 
the one hand certain data or ideas on the lar^^er 
context 0/ the field or course into whidi the teadi- 
ing unit or sequence under discussion belongs, and 
on the other data or ideas about the system of in- 
struction fields, courses, etc., whidi is to be bind- 
ing for a certain sdiool system, sdiool type, age 
level or grade. Unless we are to answer this question 
in completely amateur fashion or with uncritical 
acceptance of traditions, we are expecting too mudi 
of a number of small groups of teadiers and a 
few action researdiers practising decentralised cur- 
riculum development. Here, it is necessary to have 
information on the status of general curriculum 
theory and didactics, of the field or course under 
discussion, on the related scientific disciplines etc.; 
it would be impossible for eadi group working on 
decentralised curriculum development to work 
these out alone. For this reason, decentralised 
curriculum development must relate to supra- 
regional institutions responsible for an entire 
district, a land or a whole state, institutions whidi 
would then develop more comprehensive extensive 
general, field and course curricula (Hahmen-Hidit- 
linien) ("). However, our earlier remarks lead to an 
important consequence: it is clear that even sudi 
comprehensive curricula cannot be teadier-proof 
curricula or strict, absolutely binding lesson plans 
(Lehrplane). Then decentralised curriculum work 
could only put the final concrete toudi on pre-set 
instructions or fill out an unalterable frame. If, on 
the other hand, decentralised curriculum develop- 
ment in the above sense is to be a creative activity 
and a kind of co-determination of teachers and — 
if possible — pupils and parents on aims objectives, 
contents, organisational forms, methods and media 
of classroom instruction, then the more general 
data must have the form of open curriculum 
guidelines or open curriculum systons. "Open"" 
means here that these general guidelines must be 
criticisable and alterable; they must be regarded, 
so to speak, as normative hypotheses whidi can 
and should be put into concrete form and tested 
in the classroom and designed, diedced, dianged, 
supplemented for varying situations. A suitable 
form for sudi an interpretation, in my opinion, is 
"sdioolbased curriculum development", combined 
with action researdi. 

This concept is, therefore, in opposition to two 
contrary models on the relationship of general 



(23) See also: Deutsdier Bildungsrat, Empfehlungen der 
Bildungskommission: Zur Fdrderung sdiulnaher 
Curriculumentwidclung, Stuttgart, 1974, in parti- 
cular pages 25 ff. 



curriculum decisions on the one hand and the 
individual elements of a curriculum on the other. 

The first contrary model I call the "inductive 
model": I mean the idea that a comprehensive 
curriculum can be created by the addition of many 
individual, decentrally developed curriculum ele* 
ments. 

The second model I call the' model of '^deductive 
curriculum development". It is based on the idea 
that some central curriculum authorities — parlia- 
ments, central commissions, scientific curriculum 
institutes or whatever — can set up general 
learning aims and objectives and curricultim 
principles clearly and bindingly, but that the 
further process of curriculum development con- 
sists of deduction from that general aims objectives 
and principles until one finally arrives at concrete 
teadiing units and sequences. 

As opposed to this, the relationship between central 
and decentralised curriculum development must be 
defined differently, as a relationship of open 
interaction, as a process of constant, reciprocal 
interpretation, correction and advanced develop- 
ment. Of course, I am not describing a given 
situation, but am expressing a donand whidi is, 
in my opinion, justified. 

Why must decentralised curriculum development 
he considered a learning process? 

It has already been emphasised above, that decen- 
tralised curriculum development in the form of ac- 
tion researdi can be successful only if it is regarded 
as a learning process for all affected groups and 
organised accordingly. I will now return to this 
point. Why must decentralised curriculum develop- 
ment be considered a learning process? I will 
discuss this question using the example of the 
teadiers and members of action researdi groups. 
Both groups, teadiers and action researdiers, 
usually enter the working process of decentralised 
curriculum development with varing qualifications 
and pre-requisites. They set themselves the task of 
solving practical problems together, developing 
curriculum parts and realising them in the class- 
room an^. of critically processing the experiences 
thus gained. But, it is not all a matter of course 
that both groups regard and interpret the tasks and 
problems in the Fame way from the very beginning. 
Here are some examples (**): 



(24) Cf. Klafki, W.: ''Handlungsforsdiung im Sdiulfeld". 
In: ZeitsdiHft fUr Pddagogik, 1973, in parUcular 
pages 502 f f. — : By the same author: ''Probleme des 
Zusammenwirkens von Wissensdiaftlem und Prak- 
tikem". In: Zeitsdirift filr Pddagogik, 1973, pp. 91 ff. 
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Teadiers do not generally have sufficient training 
in scientific researdi methods. They are forced to 
teadi every day and to make constant decisions on 
short notice. They must act within a complex field 
of relationships, demands, tasks and interactions 
and assert themselves succesfuUy every day: in 
their relationships to the diildren, to their 
colleaguges, to parents, sdiool administration, etc. 
They expect co-K>peration with a researdi group to 
help them rapidly with their numerous difficulties 
in everyday sdiool life, and they hope for relatively 
quidc, noticeable improvement in the quality of 
their teadiing. 

The members of a researdi group, however, do not 
always have teadier training or practical school 
experience. But, even when they do have this 
training and experience, they are not under the 
daily pressure to act to whidi teadiers are exposed. 
This alters their perspective of sdiool reality. Their 
questions are usually of a longer term, but also 
more selective. They see only certain aspects of the 
complex sdiool reality, but these usually in finer 
detail and with more theoretical reflection than 
those involved in sdiool practice. They expect the 
form of their co-operation with teadiers and the 
results of this work to correspond as far as 
possible to scientific standards — and this means 
first of all, to theoretical standards. These varying 
perspectives — although both sides usually have 
the best of intentions and although they want to 
work on a task together — often lead to group- 
dynamic tensions, disappointments and conflicts. I 
will name two dimensions in which .^ch tensions 
very often become apparent: 

a) in the different languages spoken by the two 
groups; 

b) in the different expectations they have of 
one another. 

In both dimensions, therefore, the joint curriculum- 
development process muist also be a learning 
process: 

a) eadi group must learn to understand the 
language of the other ard try to find a 
language whidi both sides can understand; 

b) eadi group must learn to understand what 
the other expects from tliis co-operation and 
why, and they must leani to harmonize their 
varying expectations in arder to solve their 
common tasks. 

In German action researdi theory the discussion 
has often arisen that between the various partial 
groups co-operating on an action researdi project 
there must be no strict distribution of roles and 
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functions or at least that the difference in roles 
and functions apparent at the beginning must 
gradually be eliminated. I believe that this is an 
unrealistic demand in most cases. The thought 
behind it, however, is correct. I think it can be 
e>(pressed in three postulates: 

a) All members of a group of teadiers and 
action researdiers must learn to understand 
the various roles and functions to be filled 
in sudi a project. 

b) The borders between the various roles and 
functions should be kept as flexible as 
possible. For example, it would appear desir- 
able for the ''researdiers" in a decentralised 
curriculum-development project to teadi 
occasionally and for teadiers, or even pupils 
and parents, to help work on the develop- 
ment of researdi instruments, perhaps a 
questionnaire and then to use this instrument 
themselves, etc. 

c) Even when the various groups of persons 
assume differing roles and functions in sudi 
a project for practir^al reasons, these roles 
and functions must be recognised as equally 
important. 



Evaluation or use ^^f scientific methods 

These reflections, now, have consequences on the 
development of methods and instruments of re- 
seardi. 

The problem of evaluation, or the use of scientific 
methods must be re-formulated within the frame- 
work of decentralised curriculum development and 
action researdi in relation to classic empirical 
research, not only on the level of general methodo- 
logical principles, but, also on the level of develop- 
ment or application of particular methods. First of 
all, a word as to the methodological problem level: 
At least two criteria which have up to the present 
been regarded as binding for classic empirical 
researdi cannot be applied to action researdi: 

— The clear division between the researdier 
and his research instruments on the one 
hand and the researdi object on the other 
(teadiers, pupils, interaction between teadiers 
and pupils in the classroom, etc.) can no 
longer be accepted if action researdi is 
supposed above all to serve the self* 
enlightenment of all those involved. The 
researdiers must not turn the investigated 
persons and their relationships into simple 
objects of researdi. They must not, for 
instance, keep the objectives of the researdi 
or the researdi methods a secret from them. 
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On the contrary, the questions of researdi 
and — as far as possible — the researd) 
instrumernts should be discussed as often as 
possible with the teadiers or parents or 
pupils before they are applied and the results 
should be put at their disposal as soon as 
possible. I know myself, partly from the 
Grundschule-project whidi I direct, how 
difficult it is to realise this principle* and how 
far removed we remain from it in practice. 

— .'The second criteria concerns the requirement 
that the researdiers must be able to repeat 
an investigation under the same conditions 
and that in a complex field .such as classroom 
instruction only a few variables must be 
selected and studied, whereas the others 
must be kept constant over a long period of. 
time. This principle holds, if at all, to a very 
limited extent in school-based curriculum 
development and action research in our 
sense of the word. In sudi projects, it is 
diange in complex structure whidi is to be 
planned, tested and analysed. Suppose that 
a teadier->researcher group wants to develop 
a teadiing unit or a sequence of such units 
for elementary science instruction in the 
primary sdiooiP). The unit is to be related 
to the problems and to the interests of the 
diildren and their experiences with their 
environment, and the diildren are to become 
acquainted with elementary scientific me- 
thods in order to answer their questions: 
observation of animal behaviour, little 
systematic animal experiments, etc. In this 
instruction, however, they are also supposed 
to reflect on their own relationship to 
animals. With sudi a complex objective, no 
individual variables — objective variables or 
thematic variables or methodic variables — 
can be isolated and dianged whereas others 
are set, either in planning or in execution. 
And in the second nm of an analogous 
planning or teadiing process, in turn, we 
cannot rigidly attempt to keep the existing 
conditions and reject, for instance, creative 
suggestions of the diildren whidi might bring 
about a radical diange in the teadiing unit 
only in order to permit repetition of class- 
room observations under the same conditions 
as in the first experiment. 

If we accept these remarks as valid, however, we 
are confronted with a double question whidi I can 
only pose here, but not answer satisfactorily: 



(25) I refer here to a teaching unit developed by Mrs. 
Ch. Bddecker in co-operation with teachers as part 
of the Marburg Primary Sdiool Project 



a) How, then, are scientific statements possible, 
or: must the term "validity of a scientific 
statement*' be redefined? 

b) How can knowledge "gained in one place" 
be generalised or transposed to other sdiools 
and classroom situations? 

We now turn to the level of individual scientific 
processes or instruments for action research as part 
of decentralised curriculum development. Here as 
well, the ground is hardly broken. In this lecture, 
it would be impossible to discuss individual 
methods and detailed problems. Here we can only 
name a few basic aspects and difficulties arising in 
the development of appropriate research methods 
and instruments as part of action researdi; some of 
these aspects have already been briefly mentioned. 
I will name four aspects already presented in my 
paper: 

a) Tlie main questions, structure and results 
of the researdi processes should' easil;^ 
understandable to those involved. Oiily 
under this condition they can contribute to 
the common learning process of all partici- 
pants. 

b) A second difficulty in the development of 
useful researdi instruments for action re* 
seardi arises from the guiding objectives for 
development of curriculum parts. The objec- 
tives of teadiing units or sequences oriented 
to the gradual development of sell-deter- 
mination and co-determination of the diildren 
cannot as a rule be operationalised as indi- 
vidual, isolated types of behaviour in the 
sense of behavioural psydiology. Along the 
lines proposed by B. Mager(") the follow- 
ing objectives can be mentioned: 

— the cooperation ability of the children, 

— the ability to help themselves in learning 
and working processes, 

— the ability to introduce their own ideas 
and interests into discussion, etc. 

These are abilities whidi can be proved 
learned or not learned only in the context of 
certain si^^^ations and in classroom and 
extra-classroom processes; therefore, they 
can be scientifically observed only in the 
context of sudi situations and processes. But, 
who has the suitable researdi instruments? 



(26) Cf. Mager, R.: Lenuiele utmI PTogrammierter Un- 
terrichf, 14th edition, Woinhclm. 1970. — By the 
same author: Zielanalys^, Weinheim, 1973. 
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Bales, Flanders, Bellack, and others (") show 
the begmnings of scientific analysis of 
classroom processes, but, they are still too 
formal and must be developed further — 
with respect to aims and contents, as indi- 
cated above. 



c) In a type of education and instruction 
oriented to the aim and objective of self-deter* 
mina^ion and co-determination, the interest of 
research must be directed not only towards 
sudi complex aims and objectives as those 
named above, but also to the conditions and 
processes under which such "learning results'* 
are adiieved or prevented. How must class- 
room situations be structured in order for 
children to develop communication abilities? 
What kind of learning processes motivate 
diildren to develop the ability to criticise 
social prejudice? 

d) A fourth requirement of appropriate methods 
for action research has already been 
mentioned earlier, but, we must turn to it 
again in this context. If research in the 
process of school-based curriculum develop- 
ment is to offer teachers and pupils the help 
to attain such aims and objectives as 
mentioned above, then it must develop 
instruments which permit at least partly 
quick feed-back of results — or partial and 
intermediate results — to the teachers and 
pupils. Only under this condition can they, 
supported by action researchers, draw conse- 
quences from the reports for further teaching 
and learning processes during the process 
itself. Here again, we must emphasise how 
difficult it is to meet this demand, if these 
intermediate research results are to be in the 
least reliable. 



(27) Bales R.: "Die Interaktionsanalyse". In: Beobadi- 
tung nnd Experiment in der Sozialforschung, pub- 
lished by Ren6 Kdnig, 3rd edition. Cologne. 1969. 
pages 148 ff. — Flanders. N.A.: Helping teachers 
diange their behat^tor. Midiigan. 1963. — By the 
same author: Teacher inflvence, pupil attitudes and 
aMevement. Co-'Operative researdi monographt 
No. Ber 60—6, Minneapolis. 1965. — By the same 
author: In^adiievement, Co-operative Research 
Monograph, No. Ber 60-6. Minneapolis. 1965. — By 
the same author: Interaction analysis in the class- 
room, Ann Arbor. 1966. — Freeh. H.W.: KontroU 
lierte Beohachtung verbaler Verhaltensweisen im 
UnterriOit, Neue Sammlung. 1971. pages 87-108. — 
Bellack. A. et al: Die Sprache im Klassenzimmer, 
Dtisseldorf 1974. — Cf. also Koskenniemi. M.: Ele- 
mente der Unterrxditstheorie, Munidi. 1971. 
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Open and variable curriculum products 

I return to a question already mentioned: What 
nature can or must have the "result" of decentra- 
lised curriculum development created with the aid 
of action reseai^? The fact that the results cannot 
be teacher-proof curricula or teacher-proof units to 
be re-distributed as "pre-fabs** to other schools, 
have already been made clear by teachers and 
pupils. On the other hand, however, there are 
neither sufficient theoretical nor practical reasons 
for assuming that decentralised curriculum work 
in the form of action researdi cannot create any 
"products" whidi could be transposed in any sense 
to other schools, teachers, classes, and learning 
situations. 

The purpose of open, variable curriculum products 
(teadiing units or sequences) arising from decentra- 
lised curriculum development by certain groups of 
teachers and action researchers must be to stimulate 
other teachers, groups of teadiers and teacher-pupil 
groups to an analogous form of teaching and 
learning and to their own curriculum-development 
work Such "products" of individual decentra- 
lised teacher-researcher groups, that is teaching 
units or sequences or strategies of co-operative 
lesson planning, might be adapted by other 
teachers or groups of teachers under the following 
conditions: 

a) Checking and criticism of the products with 
respect to their aims objectives and pre- 
requisites must be possible. This means that 
not only formulations of learning objectives, 
structuring of subjects, etc., must be repre- 
sented in sudi a curriculum product, but 
primarily the reasons found by the creating 
teacher-researcher group for their decisions 
as to learning objectives, subjects, contents, 
methods and media, and the conditions which 
they took as a basis: for instance, the starting 
conditions for learning and the socio-cultural 
situation of the children, the teachers* own 
estimation of their abilities, etc. 

b) The products should be illustrated by 
examples, that is they should contain con- 
crete examples of discussions and learning 
objectives within the teadier-researcher 
group, for instance, or examples of pupils* 
working materials, extracts from teaching 
records, pupils* work, etc. 



(28) Cf. also: Giel, K. and Killer. G.: "Verfahren zur 
Konstruktion von Unterrichtsmodellen als Teil- 
aspekt einer konkreten Curriculumreform". In: 
Zeitsdirift filr Pddagogik, 1970, pages 739-754. — 
HiUer. G.: Konstruktive Didaktik, DUsseldorf, 
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c) The products should contain at least two to 
three practiced variants of the teadiing unit 
or sequence under discussion. As examples^ 
these variations should show how the same 
teadiing objective complex — for instance* 
**diildrens* experiences with animals and 
reflections on their relationship to animals'* 
— might be realised in various ways in the 
classroom according to differing conditions 
in the school, the class and the teadier or 
group of teadiers. 

d) The products should also open and under* 
standably discuss any difficulties and errors 
recognized during evaluation. 



Aole of the teadter in sdtool and cutriculum reform 

I have attempted to develop one basic thought all 
through this lecture: the central significance of the 
teacher in the process of sdiool and curriculum 
reform. In conclusion, one special aspect of the 
situation of the teadier in the process of decentra- 
lised curriculum development in the form of action 
researdi must yet be emphasised. Instruction 
corresponding to the aims implied above makes 
great demands on the teadiing abilities of the 
teadier. Now, this fact has one consequence whidi 
presents new problems. 

First of all, teadiers in the process of decentralised 
curriculum work must develop a high degree of 
knowledge and reflective ability with respect to 
the various partial aspects of instruction; second, 



their ability to plan lessons is increased. This, how* 
ever, gives rise to one difficulty: even outside of 
decentralised curriculum development, a consider* 
able discrepancy can frequently be found, between 
the demands teadiers make on their own instruc* 
tion — that is, their ideas of how they would like 
to teadi — and their actual teadiing practice. But, 
this does not at all mean that the actual teadiing 
qualification of the teadier automatically increases. 
Teadiers often find the tension between their own 
demands and wishes on their instruction on the one 
hand, and their teadiing reality and actual teadiing 
qualifications on the other, depressing. Therefore, 
the demand is frequently heard that curriculum 
development should be combined with new forms 
of practical advanced teadier training, with in* 
service training closely related to teadiing pratice. 
Without doubt, this is a correct postulate. But, as 
long as the present planning of open and dynamic 
teadiing units and sequences remains unsolved 
along the lines outlined above, it would be very 
difficult to attack the problem of in-service train* 
ing. In the Marburg Grundschule*project, for 
example, this double demand is a problem whidi 
can hardly be overcome. For advanced teadier 
training closely related to practice, there are 
hardly any proven methods or instruments, with 
whidi a systematic increase in teadier qualification 
for actual teadiing might be obtained and whidi 
could be practiced under normal school conditions. 
Further action-researdi projects should try to 
develop sudi methods and instruments ftom the 
very outset parallel to the methods of formative 
curriculum evaluation within the framework of 
decentralised curriculum development. 



School administration and evaluation of pilot projects 

by E. VOIGT, 

Director of the Centre of Education, Berlin. 



PART I 

The subject relates two very general concepts, to 

each other, namely sdiool administration and the 

evaluation of school reform pilot projects. 

Both concepts cover a number of quite different 

situations. 

Sdiool administration, as L. Legrand convincingly 
shows in his 1973 report, covers different functions 
and fields in the various European countries. This 
means, in my experience in the Federal Republic 



of Germany, that federal government has few 
powers, perhaps being of consequence only as a 
source of finance, while in eadi Land there h 
parliamentary responsibility and the administration 
system is organised to meet its requirements. 

The evaluation of pilot project? in different Lander 
varies with their attitudes towards the scientific 
comparison of pilot sdiools with traditional schools. 
Lander with Christian Democratic Union (CDU) 
governments demand sudi comparison and base 
further sdiool reforms on scientifically controlled 
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experience. Lin<:!er governed by the Social Demo- 
criUc ftrty (SPD) adopt the view of the majority 
of sdentiits that sudi a comparison cannot be 
carried out methodically. They also think this an 
unreasonable demand on the schools, as ii leads to 
competition wLlrh is familiar enough in commerce 
but would be unprecedented in school history. 
They have no difficulty in deciding against com- 
parison» since the changes sought are part of the 
political programme anyway. 

The administration may have various functions in 
connection with the evaluation of pilot projects: 

^ it may be interested in obtaining answers to 
questions or analysing complex situations, 

— it may be responsible for applying and adminis- 
tering public funds for such projects, 

it may ,act as the official body authorising 
questionnaires for pupils, 

— it may have to exert its authority over schools 
whidi have reservations with regard to scientific 
investigations or may have to resolve disputes 
between schools and education researchers, 

— it may be in rivalry with, or even opposition, to 
research groups in assessing disturbing factors 
in school experiments and influencing teadiers 
and the public. 

Despite this multiplicity of functions, the adminis- 
tration is obliged by its structure to r.dopt a 
relatively clear-cut and reliable attitude. It has to 
speak with a single voice, and because of its orga- 
nisation it is less affected by changes in staff than, 
for instance, project groups are. It mi t have a 
constant presence: it cannot, retire or diange its 
attitude overnight. All statements for whidi it is 
answerable to parliament acquire the force of 
medium-term commitments, at least for that par- 
liament's lifetime. The citizens are entitled to sudi 
commitments, being in any case unsettled by the 
excessively swift changes in education. But sudi 
commitments are not welcomed by education re- 
seardiers, who want their often important interim 
results to be taken into consideration immediately. 



Evaluation of pilot projectt has already become 
very heterogeneous in practice: 

— Frequently data are collected and evaluated 
in the schools, on the basis of specific hypo- 
theses, by teachers with a scientific training 
or teadiers assisted by scientists. They are 
under the authority of the sdiool and are 
also subject to the constraints of the sdiooFs 
everyday life. This is particularly effective 
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in the case of long-term, continuoxis studies. 
The studies by Teschner and Fisdiler in Ber- 
lin are examples of this type of evaluation. It 
. would be wrong to expect only concordant 
results from such groups. It is to them that- 
We owe our knowledge about the important 
selective function of options in compulsory 
education, the dangers of thoughtless action 
based on differences in adiievement and the 
prospects for heterogeneous study groups in 
the natural sciences, although their findings 
do not correspond to their original hypo- 
theses or those of their administrative 
authorities. 

— Studies by outside institutions or groups are 
more independent, but in this case continuous 
contact with the school is more difficult 
Usually such groups have the greatest 
diances of success in cross-section studies 
carried out at a specific time. But sudi 
studies are subject to the risks inherent in 
assembling data once end for all in their 
final form. In spite of its independence, this 
type of study does not present the adminis- 
tration with any problemSi even if the 
results are often unwelcome. The clear 
structure of the process (construction of 
hypotheses and collection, evaluation and 
interpretation of data) means that it can be 
used and diedced by different individuals. 

Disputes usually occur over the approval of 
questionnaires to sdiooldiildren; whenever 
ttiese are used in compulsory state schools 
they must be justified in educational and 
legal terms before the public and parliament. 

— The mixed groups of scientists and teadiers 
in more recent projects are a new pheno- 
menon whidi the administration has not yet 
completely explored. In the new ''action 
researdi"* pattern teadiers have equal rights; 
knowledge acquired concerning a field of 
action is translated directly into concepts to 
bring practical influence to bear on that 
field. I have no experience yet of the effec- 
tiveness of sudi projects as far as transferable 
results are concerned. Personal reports are 
usually rather too absorbed with detail and 
too euphoric; they remind me of the reports 
on school outings I used to submit to my 
authorities when I was a class teadier. 

Because of its function and its very nature the 
administration is bound to regard such forms with 
suspicion. It is true that this pattern is attractive 
at first sight: science comes down off its pedestal 
and exposes itself ^o the test of practice. In fact^ 
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however, it i^cquiret new powers: it brings its 
intentions to bear in interactions and feedback 
without having to defend them publicly, with the 
risks that entails (e.g. loss of an election). Often 
there are general arrangements with the adminis- 
tration which, precisely because they are so gene- 
ral, are difficult to relate to specific situations. 

The remarkably limited nature of sudi projects, 
being confined to a handful of participants and 
schools, makes action research very unobtrusive 
politically, it is true; but is it in fact realistic to 
ignore the administration, the aims of the parlia- 
mentary majority and scientific discussion? To 
apply findings directly in the finders' own field of 
action, leaving all controlling bodies (the scientific 
public and government departments) out of ac- 
count, does ind^td give them the feeling that they 
are making progress, but also a sense of isolation in 
a world which they view as increasingly unsympa- 
theUc. 

It is already clear that these mixed forms engender 
solidarity that is scientifically open to question. The 
scientists identify themselves with the "base" to 
sudi an extent that the detadiment needed for the 
acquisition of knowledge is no longer guaranteed. 
The sense of dialectic between solidarity and 
knowledge is repressed. The scientist is happy to 
be accepted. Since his function in the school is in 
any case uncertain und unnatural, he finds his 
identity by accepting for hir.iself the teadier's 
mission as something good by nature and distorted 
only by circumstances. "Teaci?r", "pupil" and 
"base" become almost sublime metaphors in the 
work of such project groups. 



PART II 

After this introduction I should like to describe 
the administration's expectations and experiences 
of the evaluation of pilot projects. First let me ex- 
plain that I have become mildly sceptical in this 
respect although I, perhaps more than others, fa- 
vour scientific researdi into sdiools and teadiing 
as well as scientific thought on the part of the 
practising teadier. My scepticism concerns the 
particular case whidi you are discusshig in this 
symposium: the marriage of sdiool projects with 
scientific interest, whidi I myself have considered 
sudi a happy one in past years. I postulate seven 
theses: 

1. Sdiool pilot projects have different values in the 
educational policies of the Lander. These values 
determine the prospects of success, and the results 
of evaluation, though this influence is not always 
realised or openly acknowledged. 
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2. In pilot projects the direct effects (those that 
can be communicated and are experienced by a 
large number of the people concerned) are more 
significant then the adiiev^ments and results of 
evaluation. 

3. The education researcher is more likely to be-^ 
come involved in conflicts of loyalty in sdiool pro- 
jects than in other educational institutions. 

4. Scientific research is bound to make for greater 
complexity. In sdiool projects this will be a disturb- 
ing factor. The inter^ts of the administration and 
the teachers demand that such complexity be mini* 
mised. 

5. School reform, pilot projects and scientific eva* 
luation have different time scales* The more clo- 
sely science ties itself to with practice, the more it 
will disappoint all concerned, by reason of its vir^ 
tually undianged temporal requirements* 

6. In practice the sidministration and teachers will 
always make selective use of the results of scien* 
tific evaluation. Thi$ is disappointing to researdi 
groups and give? rise to protests. 

7. Model sdiools cannot be exploited indefinitely for 
researdi purposes. 

Thesis 1: Sdiool pilot projects have different 
values in the educational policies of the 
LSnder. lYiese values detemUne the 
prospects of success and the results of 
evaluation, though this influence is not 
always realised or openly acknow^ 
ledged. 

Generally speaking, educational pilot projects are 
not genuine experiments. 

— In Lander which intend reforms, even sud. pro- 
jects have political significance as the outriders of 
a development whidi is desired in general terms. 
Scientific evaluation will forfeit its influence if 
with its modest instrument! and staff it diallenges 
the political premises. Its role will be to work on 
part ir«*oblems and help to clear up many of the 
questions and constraints inherent in sudi reform 
processes. 

— In Linder that seek to preserve the status quo 
because of their political majorities, pilot projects 
are often concessions to groups that are not satis- 
fied with educational policy. First and foremost 
they are show-pieces. Sudi show-pieces also exist 
in Lander that seek educational diange^ their func- 
tion being to placate conservative groups (in addi- 
tion to 6-year elementary sdiools there have been 
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in Berlin since 1952 two classical grammar sdiools 
beginning in the 5th year; however, they have ne- 
ver been scientifically evaluated). 

Progressive show-pieces raise special problems for 
the researdier, their situation being particularly 
difficult. More than in the case of other projects, 
the researdier will seek to support such isolated 
institutions and help to justify them. 

— Honesty demands-that we also mention a third 
group: pilot projects initiated in order to attract 
money to the school. Since, under the administrative 
agreement on pilot projects, the federal government 
provides funds for sudi projects in schools, these 
are often a way of raising the poor quality of sdiool 
life. As our diildren benefit from the material im- 
provements, I see nothing dishonourable in this. 

The educational researdier whom fate assigns to 
sudi a project must try to give form and meaning 
to this vague urge for progress. In view of the 
slight political significance of sudi projects they 
may prove to be a wide field in whidi to gain know- 
ledge of the subsidiary structures of sdiool life. 

Thesis 2: In pilot projects the direct effects (those 
that can be communicated and are ex- 
perienced by a large number of the 
people concerned) are more significant 
than the adiievements and results of 
evaluation. 

The social effect of school pilot projects is direct 
rather than dependent on the results of scientific 
evaluation. 

Waiting lists lor enrolment in a sdiool, the identifi- 
cation of parents and pupils with it, its "display 
v;. lue", the openness of its life to the mass media — 
all these are factors whidi generate their own dy- 
namic. An experimental school has to stand the 
test of its immediate surroundings. As an institu- 
tion with a difference it has to present its environ- 
ment with a simplified image of itself and ensure 
that this self-imposed image corresponds with what 
visitors see. In view of their progressive aims, many 
experimental sdiools thus fall into a forced rigidity 
whidi makes it impossible to deal productively with 
unexpected hitches and new developments. The 
headmaster 6f many a model sdiool finds himself 
in this situation. He is constantly having to project 
an image of his sdiool and reacts sharply if researdi 
questions this image. Evaluation of sdiool reform 
projects is researdi in a field full of tension and 
problems. 

Thesis 3: The education researdier is more likely 
to become involved in conflicts of 
loyalty in sdiool projects than in other 
educational institutions. 



Usually evaluation of sdiool reform projects covers 
individually identifiable groups. Seldom is the po- 
pulation so large that single sdiools can no longer 
be identified. If a single person or specific group 
is identifiable, problems of loyalty arise, for the 
observed behaviour of sudi a group is inevitably 
imperfect and inconsistent in some respects. I know 
of cases in whidi the school authorities and the 
immediate neighbourhood were easily able to iden- 
tify the persons referred to in the research reports. 
In the case of a large anonymous population (e.g. 
teadiers in special schools), statements based on 
empirical studies can generally be made whidi may 
be extremely effective politically without any sug- 
gestion that a particular person is to blame or has 
failed. The researdi results give rise to a discussion 
in whidi no individual is obliged to stand out. 

Evaluation thus involves crucial considerations of 
possible disadvantages to identifiable participants. 
In my experience research groups have taken this 
very seriously and an administration anxious to 
support commitment and willingness to experiment 
in its school system will welcome sudi thoughtful 
consideration on the part of researdiers. 

In recent years it has become more usual to name 
deficiencies, whUe exonerating identifiable persons 
concerned by attributing the shortcomings to ex- 
ternal factors. Studies thus end with demands for 
less teadiing time, better equipment and greater 
independence for teadiers. However, they also re- 
fer to factors about whidi nothing 'can be done, 
sudi as the parents' work or the fundamental clash 
between capital and labour. To sudi demands and 
statements, most of whidi are plausible, it xran only 
be said that they are not the result of researdi. 
There is no proof that by meeting the demands the 
shortcomings could be prevented. Sudi demands 
are a moral appendage; they result from the need 
for loyalty to an identifiable group. 

Thesis 4: Scientific re:>eardi is bound to make for 
greater complexity. In school projects 
this will be a disturbing factor. The 
interests of the administration and the 
teadier demand that such complexity 
he minimised. 

Like all researdi, evaluation of sdiool reform pro- 
jects makes for greater complexity. It shows that 
the sdiool and its projects are different from what 
was originally supposed; they are more complex 
and more interdependent. Often the results are 
completely at variance with some naive view. To 
take an example, in the sixties when in the Federal 
Republic of Germany the traditional sdiool system 
was still being projected into a stable future and 
sdiool reform was still just a subject for discussion, 
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administrations and teadiers were both astonishing-- 
ly open-minded towards the increasing complexity 
emerging from educational researdi. People accept- 
ed criticism of everything and even of themselves. 
It was a golden age for educational researdi al- 
though for lade of funds it was not possible to 
derive the full benefit from it. 

This openness dianged when reforms started; for 
the sitiuition itself became more complex because 
of the co-existence of traditional sdiools, general 
reforms (e.g. guidance periods) and a large number 
of pilot projects. Further scientific findings whidi 
reveal new factors and again diallenge ideas al- 
ready put into practice are provoking increasing 
displeasure or being ignored. The situation is wor- 
sened by the very natural professional interest on 
the part of researdiers to make a name for them- 
selves by adiieving new and revolutionary results. 

The results of evaluation will retain their influence 
only if reform processes accept a number of deci- 
sions or traditions and does not ask too mudi of 
those concerned. Any rigid pursuit of researdi that 
disregards the need for sudi moderation in its de- 
mands may perhaps bear fruit in the year 2000; but 
I predict — though you may of course disagree — 
that it will have no effect in the years immediately 
ahead. 

That is true of administrations that are confronted 
— as they often are — with factors over whidi they 
have no control. It is also true of teachers. 

Discussions in recent years have resulted in diang- 
ed standards and additional ones. While the "field 
of duty" is extending, the individual's capacity to 
achieve anything in his few working hours is still 
basically the same. The incompatibility between 
more complex norms and imperfect everyday life 
presents job satisfaction; indeed, it makes for sus- 
ceptibility to ideas that offer hope of relief. 

Thesis 5: Sdiool reform, pilot projects and scien- 
tific evaluation have different time 
scales. The more closely science ties 
itself to practice the more it will dis- 
appoint all concerned, by reason of its 
virtually undianged temporal require- 
ments. 

This thesis does not say anything new. When re- 
seardi takes place remote from practice and the 
daily need for decisions, sudi differences do not 
become a public problem. The closer researdi co- 
mes to practice, the more it arouses hopes of a 
secure basis for decisionr and the less fitted for 
this it comes to appear to practising teadiers be- 
cause of all the time it requires. Researdi groups 
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whidi often co-operate with practising teadiers 
must find this reproadi, whether explicit or im- 
plicit, discouraging. 

The impression that the evaluation of sdiool reform 
projects does not take time suffi:iently into ac- 
count is strengthened by the considerable amount 
of time whidi the project groups take to settle 
their internal problems. Sudi a group is a new 
social phenomenon of whidi we have very little 
experience. Its social equilibrium is very un.^lubL*. 
it tends to the formation of splinter groups and 
gives rise to many internal problems! There are 
times when it is largely inward-looking. Such time- 
consuming forms of decision-taking make a ter- 
rible impression on teadiers working time-tables 
imposed by others. Indeed, the different time-pat- 
terns of scientists encourage the feeling in the 
administration and among teadiers of a difference 
in the very quality of life. 

Thesis 6: In practice the administration and 
teadiers will almost make selective use 
of the results of scientific evaluation. 
This is disappointing to researdi groups 
and gives rise to protests. 

Administrations and teadiers are always being ac- 
cused by researdiers of selecting for practical im- 
plementation only these researdi results that suit 
them, and of never taking into consideration the 
findings as a whole, with all their "ifs" and "buts-. 
This reproadi is only partly justified. Nobody no- 
wadays can suppress really serious objections, 
even if he wanted to. 

I refer here to the justified criticism that a person 
who has to take a decision makes a selection, even 
if only, unconsciously. In my view this cannot be 
dianged, and the strategy of evaluation must take 
account of the fact. The progress in education over 
the last centuries is not the result of implementing 
ideas and scientific findings in their entirety. In 
every case only a few factors had any historical 
effect. The premisses and the context, everything 
that was important to the educationalist and the 
researdi worker, were soon forgotten. 

Thesis 7: Model sdiools cannot be exploited 
indefinitely for researdi purposes. 

In recent years the administration has increasingly 
often found itself obliged to mediate between mo- 
del sdiools and research groups, usually because of 
the unwillingness of sdiools to be subjected to re- 
seardi any longer. Last year in Berlin the Senator 
had to use his authority to ensure the implemen- 
tation of another sdiools research project. Chris 
Argyris once made the pointed observation, "For- 
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merly the barons of the timber trade had trees 
felled without bothering very mudi about the future 
of the industry. Researdiers use persons for experi- 
ments without bothering very much about the fu« 
ture supply of sudi people" (in "Gruppendynamik", 
1972. Vol. 1). 

Since moreover many students carry out empirical 
researdi for their state examinations and degrees 
and find educational pilot studies especially attrac- 
tive for the purpose, it falls to the state to regulate 
the pressure of researdi on the schools. 

The ideas I have put forward are the result of 
limited experience, and because of my job. are 
one-sided. I nevertheless draw the following con- 
clusions from them: 

— Evaluation is not just a problem of researdi me- 
thods. That became clear in 1970. when a large 
number of conflicts of interest and misunderstand- 
ings arose between sdiools. administrations and 
scientists. 



— These disputes cannot be solved through systems 
of co-operation alone. There has been a clearing 
commission for sudi disputes in Berlin since 1972. 
That is a step forward. But even so there is a dog- 
fight of expectations and interests because those 
concerned have no time to give thought to evalua- 
tive researdi as an instrument with all its opportu- 
nities and limitations. It is just not enough simply 
to get people with different interests around a table, 
however essential that may be. 

— Evaluation also calls for realistic ideas on stra- 
tegy. Special consideration must be given to: 

• the Land's educational policy. 

• the project's stability or instability. 

• reducing complexity. 

• the establishment of a realistic time-scale. 

• the limit to the demands that may be made on 
sdiools. 



From centralised to decentralised Innovation - 
Problems of evaluation 



The process of educational innovation — central- 
ised as well as decentralised — may be regarded as 
a continuous transition through distinct stages, 
eadi of whidi involves dissimilar objectives, tasks 
and priorities. This holds for wide ranging edu- 
cational reforms, as well as for specific projects. 

These stages may be broadly diaracterised as: 

— Planning 

— Development 

— Follow-up 

Evaluation is seen as a vital element at eadi stage, 
if central, regional and local dissatisfaction with 
"ad hoc", random experimentation is to be avoided. 



The tradition of centrally sponsored research 

A strong tradition of political influence in educa^ 
tional planning is typical in those countries which 
possess a more or less monolithic school structure; 
In sudi countries, there has been a corresponding 
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tendency to rely on central political and admini- 
strative initiative for educational development. 
Where this is the case, it has fallen to central au- 
thorities to ensure evaluation of such sdiemes as 
have been put into effect. However, evaluation has 
not traditionally been considered an integral part 
of the process whereby funds are allocated for 
educational research. The bodies responsible for 
financing or sponsoring researdi projects have 
played a largely "passive" role. It has been 
taken for granted that once funds are allo- 
cated, it should be left to professional researchers 
to tadcle the actual work involved in their own 
way. and to return reports and accounts on com- 
pletion of any given project. Such studies have 
been largely carried out at universities under the 
leadership of established and experienced resear- 
diers according to relatively narrowly defined cri- 
teria. 

Recent years have seen a distinct tendency towards 
direct and active involvement in the researdi pro- 
cess on the part of the sponsors themselves. Sudi 
involvement has included the circulation of infor- 
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mation acquired as well as distribution of mate- 
rials developed in the course of projects. 

Manifold educational reforms over the last two 
decades — organisational, structural and curricular 
— have prompted a political initiative from both 
inside and outside the educational system, leading 
to the creation of bodies specifically responsible 
for systematic evaluation of research projects. 
These trends were particularly apparent during 
the sixties. The resources allocated to these bodies 
were directed mostly towards policy orientated 
researdi, while the authorities themselves showed 
a growing interest in researdi findings. 

Official interest has no doubt been prompted by 
a felt need for assistance when making decisions 
on complex educational matters. 



Problems oj centrally siponsored researdi 

Centrally sponsored researdi, of the kind sketdied 
above, has encountered a variety of problems, of. 
which the following should be briefly mentioned 
as being relevant to attenpts to organise reseavdi 
on a decentralised basis. 

— The monitoring of projects leads, too eafaly, to 
interference in the conduct of the projects. 

— Whenever researdi funds are allocated by agen- 
cies whose own researdi policy is cl'^ar cut, the 
conduct of the researdi itself tenos to be dic- 
tated by those agencies rather than by the re- 
search staff themselves. 

— It is frequently difficult to ensure adequate 
financing of researdi directed towards 

' • critical evaluation 

• alternative models 

• evaluation of underlying basic assumptions. 

— Central authorities have tended to give prio- 
rity to short-term projects dealing with imme- 
diate practical problems. 

— Funds are often allocated on a year to year 
basis, owing to the unwillingness of sponsors 
to commit themselves to supporting research 
projects until they are fully completed. 

— It has, to some extent, been difficult to draw 
on the most experienced and best qualified re- 
search expertise for applied research. 

Nevertheless autorities have in general attempted 
to create an environment in which the findings of 
educational research can influence educational po- 
licy as well as teaching. Authorities must be pre- 
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pared to take some risks if such influence is to be 
really effective. Countries with centralised educa- 
tional structures have not yet had sufficient time 
to adiieve wholly satisfactory solutions to the pro- 
blems whidi have arisen during the course of this 
process, or to establish a sufficiently effective 
relationship between research and educational po- 
licy making. It is, in any event, certain that the 
effect of research findings on the decision making 
process in education must depend on the existence 
of an appropriate system for diannelling relevant 
information. 

The problems involved must be faced squarely, 
tadcled seriously and, where necessary, unconven- 
tional solutions to them must be examined. 



The experience of decentralisation in Norway 

Recent experience ia Norway may, perhaps, illu* 
strate some of the problems. In Norway, it has 
become obvious that centrally directed attempts at 
researdi into educational questions are frequently 
doomed to frustration. There is now an increasing 
tendency to move away from centrally controlled 
reform and towards involving various types of re- 
gional and local institutions at different levels. Po- 
licies directed towards decentralisation are also 
characterised by a determination to employ more 
resources in the solution of educational problems, 
rather than towards supplanting central resources. 
Until recently, uniform educational structure and 
political determination have been regarded as the 
most important pre-conditions for raising the ge- 
neral level of educational opportunities for people 
from every section of society 

In the late sixties and early seventies some coun- 
tries have begun to feel that the level of general 
education is sufficiently high (or is in the process 
of becoming so) for local or regional initiative to 
be relied on to develop further differentiated edu- 
cational opportunities for groups and individuals 
in eadi area. In Norway for example, the state, for 
its part, has maintained financial support for the 
various types of school that have been established. 
In place of detailed instructions for^ for example, 
standard curricular within a uniform system, cen- 
tral authorities are increasing favouring indicative 
norms. At the same time it is admitted that the suc- 
cess of this policy must, to a large extent, depend on 
local and regional initiatives and the active parti- 
cipation of teadiers, parents and pupils in the 
organisation and provision of educational choices. 
Such initiatives should also be able to contribute 
to the establishment of special courses beyond, and 
in addition to, those that already exist at various 
levels in the school system. 



30 



The execution of any decentralising policy of the 
kind referred to must obviously take some time. 
Some years have elapsed since it was initiated, and 
several more will pass before the process is com- 
pleted. 

It will be necessary to establish and define unfa- 
miliar responsibilities for a range of officials — 
and in some cases new posts will have to be creat- 
ed to cope with these. A variety of new bodies must 
likewise be established to carry out the tasks, eg 
consultative councils, education centres with their 
teams of advisers. 

New and established institutions sudi as district 
colleges, colleges of education etc, must adjust 
themselves to new tasks. 

Modern western society seems to be approadiing 
what one might call an epodi of ^'enlightened de- 
mocracy". Universal education and general affluen- 
ce offer society opportunities to tackle tasks in new 
ways and in new contexts. Centralised systems 
developed sudi structures as were necessary to 
permit them to function according to their own 
lights. Decentralised systems must do likewise. 
Transition from centrally to locally organised in- 
novation typifies one area of activity calling for 
re-structuring and delegation. 

Unless the political will for decentralisation is 
backed up by the appropriate executive structure, 
financial responsibility is delegated, and qualified 
personnel is appointed locally, there will be no 
foundation for local initiative and enthusiasm to 
rest on. Enthusiasm and optimism at local level 
must be based on a realistic appraisal of conditions, 
if it is to contribute towards a favourable environ- 
ment in whidi decentralised reform can take place. 



Decentralised innovation: the planning stage 

A common objective for all educational reform is 
the qualitative improvement of educational oppor- 
tunity. It may, of course, be questioned whether 
sudi an improvement can be adiieved at all through 
decentralisation. A variety of factors will determine 
the answer to that question, not least the quality 
and the level of the planning undertaken. One may, 
nowadays, be permitted to assume that local and 
regional authorities and administrative bodies are 
familiar with general principles concerning educa- 
tional planning. However, while centralised plann- 
ing by and large takes account of concerns that are 
common to all regions of the country, local and 
regional planning can more easily take into consi- 
deration needs that may be peculiar to the educa- 
tional environment of the local community con- 
cerned, and even to particular institutions .vithin 



that community. It is nevertheless vital to ensure 
that those responsible for applying general prin- 
ciples to specific situations are properly qualified 
to do so. If sdiools are to be run on democratic 
lines ti is obvious that planning — whether central 
or local — must not be left in the hands of an 
elite. Experts must be regarded as members of a 
team, some of whose expertise consists of thorough 
knowledge of local conditions — needs — concerns 
— and resources. The value of any proposals or 
plans drawn up by sudi a team will thus lie in the 
fact that they will be specifically tailored for the 
particular environment in question and in many 
cases for that environment alone. Freed from the 
need to take into account matters of general con- 
cern, local and regional planning can concentrate 
its attention in the opposite direction, namely on 
matters that are of purely local concern. 

Centrally sponsored reform has always tended to 
stifle local efforts. The creation of a nationally 
uniform sdiool system led to a great number of 
serious problems; as when attempts were made to 
establish state sdiools in various districts, eadi 
-A^ith its own special diaracteristics. Most of these 
problems should be soluble within the context of 
decentralised reform. 

As far as educational researdi is concerned, decen- 
tralisation should offer clear advantages insofar as 
it will allow for a systematic analysis of existing 
local conditions, collaboration of shortcomings, and 
suggestions for possible alternatives. At this stage 
it is important to consider reports and discussions 
of completed as well as current development pro- 
jects, both at home and abroad. The aim here will 
not be simply to note broad conclusions or findings, 
but to study the conduct of the various projects in 
similar environments, in order to discover parti- 
cular problems and findings related to them, that 
concern the type of environment in question. Sudi 
information is of particular value in decentralised 
reform, and should provide researdiers with in- 
timate knowledge of the local environment from 
the outset of a project, while at the same time fa- 
miliarising them with matters relating to proce- 
dure. 



The development stage: continuous formative 
evaluation 

Reform at local level, aimed at meeting the in- 
terests of the locality, calls for continuous evalua- 
tion. Continuous formative evaluation will aim at 
discovering the extent to whidi sudi processes as 
have been established, or put into effect, seem 
likely to contribute to stated objectives — whether 
these are of a broad, long-term, or more limited 
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nature. The objectives of evaluation at this stage 
are twofold: 

— to ensure that what is in fact carried out con- 
forms to stated intentions; and 

— to provide an objective basis for subsequent 
decisions. 

The likelihood of adiieving both of these objectives 
will be improved if the researdi staff involved are 
themselves associated with regional or local insti- 
tutions — eg colleges of education, district colleges, 
educational service centres etc — in this way it 
will be possible for researdi staff to ensure con- 
tinuous appraisal of developments through direct 
observation, direct discussions and seminars involv- 
ing-other interested parties. It should accordingly 
be unnecessary to rely on procedures that are more 
appropriate in situations where contact between 
those involved in the process of reform and those 
evaluating it is remote and intermittent. 



The influence of research on political and 
administrative decision making 

Decentralised evaluation may be considered pri- 
marily as an intelligence service within the orga- 
nisation of continuous reform (whidi may inciden- 
tally form an actual part of the education service 
provided by the local community). In this way it 
is more likely that the contributions made by re- 
search staff themselves will keep pace with deve- 
lopments and» furthermore, will be available when- 
ever decisions must be taken* Administrators are 
often unwilling or unable to devote time to study- 
ing lengthy reports or exhaustive documentation. 
It should be possible for them to become acquaint- 
ed with the various contributions of researdi in 
other ways — eg through discussions focused on 
those areas within whidi specific decisions have to 
be made. If researdi — development — dissemina- 
tion — implementation — administration — and 
policy making — are to be fully co-ordinated, it 
will be necessary to establish far more effective 
and fluid lines of co«Timunication than has hitherto 
been the case. 

Perhaps the greatest advances are yet to be made 
at the level of personal contact — whidi is in any 
case obviously more feasible in the context of 
decentralised (as opposed to centralised) reform. 

Various kinds of researdi — development — dis- 
semination - innovation and institutionalisation 
have perhaps been more satisfactory in social and 
economic contexts than in education. We are aware 
of few projects that are directly applicable to actual 
teadiing, or to educational administration. 



Headmasters, and principals of schools and colleges 
have looked in vain for guidance to major researdi 
projects dealing, for example, with classroom dif- 
ferentiation. Administrators and authorities with 
budgetary responsibility have found no clear gui- 
dance from researdi dealing with, for example, 
optimum class size. It is indisputable that a great 
number of people associated with the sdiool have 
found the results of educational researdi singu- 
larly unhelpful. 

It is nevertheless important to recognise the poten- 
tial influence of researdi findings on political and 
administrative decision-making in the context of 
decentralised educational innovation. Sudi in- 
fluence may be detected in the general political 
climate and more specifically in attitudes regard- 
ing matters of educational policy, insofar as actual 
decisions are felt to be founded on rational criteria. 
Provided that the quality of researdi can be main- 
tained at a high level, and that the bulk of the 
researdi findings can be co-ordinated, it is reason- 
able to suppose that varied findings will never- 
theless display broadly discernable common tenden- 
cies. These will, in turn, contribute towards Ihe 
establishment of the kind of climate mentioned 
above, and will, of course, help to determine de- 
cision-making. Researdi may have more to offer in 
terms of general indications, as apposed to spe- 
cific concrete conclusion. It should not be a func- 
tion of researdi to determine political or executive 
decisions, nor to dictate to, or undermine the autho- 
rity of executive bodies and administrators. 

Education shares with politics a broad concern 
with life style. Eadi involves highly complex con- 
siderations whidi cannot be supplanted by educa- 
tional researdi on the one hand or political science 
on the other. Science and researdi can help to 
foster a more rational approadi without necessa- 
rily making the decision-making process any easier. 



The follow-up stage 

While innovation should form an integrated com- 
ponent of contemporary teadiing practice it is 
desirable to define eadi new element in sudi a 
way as to ensure continuity within the existing 
framewo.^k — and to preserve a clear distinction 
between temporary expediency and standard prac- 
tice. Continuous formative evaluation must be 
complemented by summative evaluation upon com- 
pletion of any programme of innovation. Summa- 
tive evaluation should offer a basis for decisions 
as to whether, and in what form, innovation is to 
continue: as well as provide pointers to possible 
improvements in existing practice. The relation- 
ship between evaluation and decision-making at 
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this stage will roughly correspond to that indicated 
above in respect of earlier stages (planning and 
development). 

In this context, too, it is important for research 
staff to be in sufficiently close contact with the 
local community for them to be able to convey 
researdi findings to those responsible for executive 
decisions. Their task at this stage will be not so 
much to generalise from research findings, as to 
concentrate on the significance of those findings as 
far as the reform in question is concerned. 



The division of responsibility between centre 
and periphery 

During a period of rapid educational growth and 
diange, it is reasonable to expect curriculum de- 
velopment to be given the "lion's share** of avail* 
able resources (staffing and financial). This is the 
case at present. The best work in curriculum de- 
velopment will be solidly founded on relevant learn* 
ing theory, advanced social psychology, and know* 
ledge of behavioural dynamics — in the sdiool, in 
the classroom, and in other groups at various age 
levels. 

Not all local environments will be equally well 
qualified to undertake this kind of work. More- 
over, there is a growing recognition of the fact that 
work on curriculum development in general seems 
to suffer from weak theoretical fondations, that is 
to say that fundamental theoretical researdi has 
not kept pace with the recent flurry of applied 
research and the expansion in education generally. 
The trend towards decentralised reform makes pos- 
sible a more sensible differentiation between va- 
rious types of institutions and other bodies. No 
single institution can claim to be wholly equipped 
to tackle the myriad of dissimilar tasks that come 
under the heading of "educational innovation**. The 
universities should be permitted and indeed, en- 
couraged to continue to develop their basic theo- 
retical expertise, while institutions more concerned 
with practical teadiing — eg colleges of education, 
educational service centres etc — should direct 
their attention to correspondingly practical tasks. 
Responsibilities should be allocated to the various 
administrative bodies in a similar way. There will, 
of course, always be some institutions and admini- 
strative bodies whidi fall between the two extre- 
mes (theoretical and practical) whose proper func- 
tions will frequently and appropriately be to ensure 
contact between the two extremes, to bridge any 
gaps between the theoretical and practical — as 
well as between centralised and decentralised in- 
novation. 



Centrally directed innovation should be primarily 
concerned with "macro-level** issues, ie those re- 
volving around sudi considerations as — "the role 
of education in society** — „the structure and func- 
tioning of the educational system in relation to the 
goals set for it and the resources allocated to it**. 
Local and regional bodies will of course, take sudi 
issues into consideration — albeit within more 
defined limits — to the extent that they are finan- 
cially autonomous. 

Decisions at "micro-level" — where the emphasis 
will be on sudi issues as "the development and 
characteristics of the individual student** — "prac- 
tical conditions for efficient learning** — etc, should 
be taken by local authorities. It should be pointed 
out here that collaboration between local and cen- 
tral «iuthorities should take place to a greater ex- 
tent than hitherto. 

Between the two extremes there are intermediate 
level tasks where matters relating to the learning 
environment, the curriculum and so on are of cen- 
tral importance. It is possible that just such tasks 
can form the basis of concrete projects in whidi 
institutions of various kinds and levels co-operate 
with eadi other. 



Questions for decision 

Important tasks io be tackled during the transition 
from centralised to decentralised innovation are: 

— defining and categorising the specific problems 
that should be researdied; 

— attempting to establish or identify the type of 
institution that is most suitable for coping with 
the problems; 

— deciding the level at whidi the problems should 
be approadied. 

Such procedures should be appropriate and feas- 
ible in the context of evaluation in innovation as 
sketched above. 



Research priorities 

One example of a plan for research work is that 
drawn up by the Rand Corporation for the Natio- 
nal Institute of Education in the USA. The plan 
defines four areas in whidi renewed effort is 
called for. Eadi area is further divided into a 
number of "programme elements". The first of the 
areas covers such major educational problems as 
the improvement of educational opportunities for 
the disadvantaged, improvements in the general 
quality of education and the achievement of grea- 
ter efficiency in the use of educational resources. 
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The second area alms at improving educational 
practice while the third aims at strengthening the 
foundations of education — selective researdi pro- 
grammes building basic knowledge concerning 
education. The fourth area is entitled "Strengthen- 
ing the researdi and development system". A fur- 
ther step would be to work out a plan of what 
institutions and levels should have the main re- 
sponsibility for the different* areas or programme 
elements. 

Another example of an attempt to establish prio- 
rities for researdi tasks may be found in the Nor- 
wegian Department of Education's Statement of 
Intent regarding educational researdi. The depart- 
ment has suggested that researdi should be carried 
out in the following areas from 1975 onwards: 

— the ways in whidi the\new Norwegian outline 
curriculum ("Mensterplan**) is being put into 
effect in primary and lower secondary sdiools, 

— integration of handicapped diildren into the ge- 
neral sdiool system^ 

— developments in sdiools at upper secondary 
level, 

— the place of the school in the local community, 

— analysis of the effects of decentralised executive 
authority within the education system, 

— evaluation of high cost teaching materials, 

— investigation of the relationship between special 
education and opportunities for the physically 
and mentally handicapped in society, 

— the roles of the sexes in education, 

— identification of groups availling themselves of 
opportimities in adult education. 

It is open to any institution, group or individual 
to apply to the Department of Education for full 
or partial financial support for projects within 
these areas. The Department of Education itself 



has taken active steps to draw local institutions 
into this work, illustrating one way in whidi cen- 
tral authorities can tadcle problems in whidi they 
themselves are interested, at the same time as they 
stimulate local interest in researdi evaluation. But 
so far we have derived little experience from the 
involvement of various institutions at different 
levels in solving problems within broadly defined 
researdi areas. 

The Norwegian Department of Education also sup- 
ports researdi in other ways, eg through grants to 
central organisations sudi as the Norwegian Coun- 
cil for Innovation in Education, the Norwegian 
Researdi Council for Science and the Humanities, 
as well as local bodies. At present, projects initiated 
locally must, to a large extent, rely on local finan- 
cial resources. However, work'iig parties on edu- 
cational researdi are now preparing proposals for 
co-ordinated programmes whidi will incorporate 
plans for financial support for educational researdi 
carried out at all levels — local, regional and cen- 
tral. 



Conclusion 

It appears obvious that problems associated with 
the trend towards decentralised innovation and 
evaluation are not simply financial, organisational 
or administrative. Neither decentralised innovation 
nor evaluation are likely to be wholly successful 
unless the institutions and individuals involved 
(research workers, teadiers, administrators or local 
people) are given broad institutional support, as 
well as theoretical and practical training. At the 
same time it is to be hoped that improvements in 
the quality of education provided in eadi locality 
will contribute towards the establishment of a 
political climate whidi will in turn ensure conti- 
nued development. 



Local variations in national curriculum development 
projects: self-generated school-based projects 
and their implications for evaluation 



I want to begin this talk in a slightly unorthodox 
way by referring to a minor error in the documen- 
tation.. You might have noticed that one document 
refers to badcground information from the "U.K.", 



by F. H. SPARROW, 
Sdiools Council, London. 



whidi means, of course, the "United Kingdom". If 
any of you have followed the events of last week, 
particularly the General Election last Thursday, 
you might consider that the adjective "United" is 
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open to some question. If this constitutes an "error", 
it is not the one to whidi I refer. The information 
comes from England and Wales and I must make it 
clear that I will speak only of these. One of the 
things which England has in common with Scot- 
land is a different educational system and I have 
no brief or competence to speak for Scotland. In 
my talk I will probably refer only to England, but 
this will be purely for the sake of brevity and is 
not to be taken as a gratuitous insult to Plaid 
Cymru, the Welsh Nationalist Party which won one 
notable victory over the British Labour Party last 
week.. 

This talk covers six main areas. 

— A brief resume of the, English system in the 
light of the title of this symposium. 

— Some illustrations of how curriculum develop- 
ment grows from and is influenced by this 
system. 

— Some aspects of evaluation trends. 

— Reasons why the Sdiools Council is currently 
considering support for self-generated, sdiool- 
based projects. 

— Some examples of local projects which are being 
considered for support. 

— Possibilities for future development and their 
implications for evaluation. 



The English system 

The essential point whidi has to be stressed is that 
it is de-centralised. The Department of Education 
and Science is the official Government Department 
and it has a great deal of power. It administers the 
law and it naturally plays a leading role m the evo- 
lution of new laws and new policies. It has consi- 
derable control over funds, a control whidi is, per- 
haps, most apparent in respect of sdiool buildings. 
Her Majesty's Inspectors act as the eyes and ears 
of the Department. 

We also have over 100 Local Education Authorities, 
a fact which surprises many Germans, v/ith their 
eleven Lander and Swiss, with their 22 Cantons. 
Indeed, until last April we had over 160 of these 
Authorities. Eadi has a great deal of power. It is 
responsible for implementing the law within its 
own area. It handles the money which comes from 
local sources in the form of rates. It submits sche- 
mes for Comprehensive Sdiooling to the Depart- 
ment and in this connection it is interesting to note 
the extensive range whidi exists. We have Compre* 
hensive Sdiools for the ages 11 to 18, 11 to 16 
(followed by open-entry Sixth Form Colleges), 12 



or 13 to 16 or 18 (where there are Middle Sdiools 
for diildren up to the lower age) and so on. Indeed 
the confusion over terminology to whidi Professor 
Mitter referred is very apparent in England. The 
Local Authority maintains sdiools, with their 
buildings, equipment and staffs. It employs Local 
Inspectors who are commonly called "Advisers" 
these days. 

We also have thousands of sdiools, eadi of which 
has a great deal of power. The most powerful sing- 
le figure is the Head Teadier, who has been re- 
ferred to as ''the last autocrat in England**. Nowa- 
days there is an increasing trend towards a hierar- 
diic structure of Deputy Heads, Year or House 
Tutors and Heads of Departments. Even so, the 
most important person in the last resort is the 
classroom teadier and he too has a great deal of 
autonomy. In effect, nobody, whether from Depart- 
ment, Local Authority or School Office can tell 
him what to teadi or how to tcadi it. In case you 
think that this must be a myth, let me illustrate 
from my own career. I taught in England for 
twenty years in many different types of sdiool. In 
all this time I saw Her Majesty's Inspectors on only 
six occasions. I was never visited by any Local 
Authority Adviser and no Head Teacher ever heard 
me teadi. It is a fact that nobody ever told me 
what to teach or how to teadi it. 

From this it will be clear to you that if we are 
talking about a general trend towards de-centrali- 
sation England is already more than half-way 
there. However, I must add a caveat. There are in 
fact many iiifluer :es at work whidi give the sy- 
stem more cohesion than appears at first sight. 
English teachers come from what might be described 
as the same gK eral milieu. Although there are 
certainly wide regional variations there is also a 
lot in common in the educational ethos and back- 
ground. Moreover, many teachers experienced a 
similar sdiooling, some aspect at least of which 
they perpetuate in their own teadiing. At its worst, 
when a teadier has been educated in a very tra- 
ditional school, followed by a formal University, 
this can lead to what has been called "the cycle 
of stagnation". Another influence is the frequent 
meetiiigs of teachers, ranging from the informal to 
Courses and Conferences arranged by many pos- 
sible agencies, including HMI, LEA Advisers, Pro- 
fessional Associations, Subjects Associations, Uni- 
versity and Colleges and Examination Boards. Eng- 
lish teadier s work less in isolation than ever they 
did. Another centr£:lising influence is the external 
examination system, about whidi a lot could be 
said in another context. The situation is, of course, 
much more complex than so brief an account can 
possibly reveal, but my intention is to demonstrate 
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that curriculum development can take place in 
England only if it is undertaken in ways whidi 
neither threaten nor diallenge that autonomy whidi 
is sacrosanct. 



Illustrations of English currtculum development 

In the early 1960*s the Department of Education 
and Science made a move to stimulate curriculum 
development by setting up a central Curriculum 
Study Group. The result of this was an outcry 
from the teadiing profession. It appeared to them 
that their cherished autonomy was about to be 
overthrown and that a central authority would 
soon dictate the curriculum to the schools. How- 
ever true this was — and I, as a teacher, was cer- 
tainly convinced of its truth — the teadiers made 
it clear that they would have no part of it. The 
result of this was eventually the setting up of the 
Schools Council, the organisation whidi I represent 
here today. Most of the curriculum innovation 
initiated in England since 1964 has been under its 
auspices It is a central organisation whidi works 
in a decentralised and non-directive manner. The 
point can be illustrated by reference to various 
aspects of Council Projects. 

The decision whether or not to sponsor projects is 
made by policy Committees, all of whidi must, 
under the constitution, have a majority of serving 
teadiers. In this way it is the teadiers who decide 
wnat is to be undertaken. The projects are based 
on centres spread almost throughout the land, so 
that although the sponsorship is central, the pro- 
jects are widely dispersed. Project teams normally 
include serving teadiers who are released from their 
teadiing duties for the relevant penod of time. 
Methods and materials are tried out in sdiools 
whid: are diosen not for thdr assumed excellence 
but for the way they represent sdiools as a whole. 
The sdioois are approadied only through their Lo- 
cal Authorities. No school and no teadier is obliged 
to co-operate in project work. Teadiers who take 
part are intended not to be merely passive recei- 
vers of padcages, but active partidpants in a com- 
mon enterprise. Methods and materials are never 
intended to be definitive or prescriptive. It is as- 
sumed not only that teadiers will feel free to select 
material and add their own, using methods whidi 
vary according to their perception of their children's 
needs, but also that this teadier freedom is good in 
itself. When projects have completed their trial 
stage and published reports, teadiers* guides and 
pupil material are available, sdiools are completely 
free to accept or reject what is offered. The Coun- 
cil's aim is not to impose curriculum development 
on anyone, but rather to widen the range of dioice 
available to teadiers. 



There is always a danger that speakers in my pre* 
sent position will rationalise the status quo; that 
having to accept situations which they cannot con- 
trol, they will make the theory sound good, no 
matter what the reality of the practice. In case you 
think that what I have said about projects reveals 
self*delusion I can assure you t!iat if we took that 
long list of projects whidi ycu have been given, 
I could show you how each one illustrates what I 
have said. In the limited time available I can only 
use examples. One of these projects, children as 
readers, has no defined central programme. It sti- 
mulates groups of teachers to conduct their own 
development work. The history, geography and so- 
cial science project starts from a number of core 
concepts, but it does not intend to produce mate- 
rials for all teadiers so mudi as to assist teadiers 
to develop their own, tailored to the needs of iheir 
diUdren in their situation. The continuing maths 
project produces no pupil materials centrally, but 
a large quantity of material is produced by locally- 
based groups of ^adiers, with help and support 
from the central team. The materials produced 
centrally by the humanities curriculum project 
have sold well, but it is estimated that its users 
fall into three categories, one 'of which applies the 
methodology of the project as closely as possible, 
another uses the material as invaluable si:pport for 
teadiing in ways whidi owe nothing to the project, 
while a third probably mixes the project philosophy 
with various preferred teadiing styles. 



Some aspects of evaluation trends 

It follows from all I have said up to this point that 
the evaluation of curriculiun development in Eng- 
land cannot possibly be a simple process; indeed, 
as eadi evaluation is geared to a project rather than 
to some abstract external model, no two of our 
evaluations are alike. However, after over ten 
years* experience it is possible to detect certain 
trends whidi reflect the decentralised English sy- 
stem. For example. Her Majesty's Inspectors were 
used extensively in early evaluation. This was in 
accord with a long-standing tradition. Although it 
csn reasonably be argued that sudi evaluation is 
subjective, it must also be remembered that these 
Inspectors were in a good position to compare 
sdiools and that their pooled information repre- 
sented more than a narrow personal judgement. In 
any case, it is probably wise occasionally to remind 
ourselves that evaluation means '"valuing" and that 
in the last resort this must be largely personal, no 
matter how objective the evidence may appear to 
be. When the practice grew of appointing evalua- 
tors or evaluation teams to Sdiools Council pro- 
jects there was a very strong tendency for these 
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first "professionar evaluators to be influenced by 
the engineering model. The stress tended to be on 
measurement: pre* and post-tests, control groups 
and the like. In the last four or five years there 
has been a distinct movement away from this ap- 
proadi. There are many reasons for this. The Eng- 
lish scene whidi I outlined earlier is hardly amen- 
able to exclusively measurement procedures. There 
is no centrally determined curriculum, the various 
levels of autonomy I described have to be respect- 
ed, there is no attempt whatsoever to produce 
"teadier-proof** materials and the teadiers them- 
selves are closely involved in the development pro- 
cess. In sudi a context the stress has had to be 
increasingly on local variations rather than on 
national measurement. Perhaps England is not as 
unique as I appear to be suggesting and that what 
Dr. Wynne Harlen has recently written for a book 
we are hoping to publish next summer will find an 
echo wherever there is curriculum development. 

"When put into practice this classical model, whidi 
seems logically attractive, fails to give the infor- « 
mation whidi is most useful either for formative 
or summative evaluation. What makes it so attrac- 
tive and reasonable is precisely: its weakness; it 
is part of an oversimpliffed view of curriculum 
development as 'identification of objectives, plann- 
ing of learning experiences, evaluation* (Wiseman 
and Pidgeon, 1970), which ignores the characteri- 
stics of the learning environment and the inter- 
action of learners and teadiers with it, both in the 
development and evaluation stages. In reality what 
pupils learn in any situation depends on a complex 
collection of factors which have to be taken into 
account if the evaluation information is to be of 
practical value." 

It is certainly the case that evaluation in England 
has concentrated increasingly on "process** rather 
than "product**. We want to know not so mudi the 
estimated total gain but more the context in whidi 
certain results are found. The number of variables 
which affects these outcomes is very large indeed. 
Our evaluators collect as mudi data as possible 
about them, using a wide range of the techniques 
available. In addition to organising feed-bade from 
diildren and teadiers, some of them use tests of 
attitude (tending to use scales developed for the 
purpose as existing oner are rarely suitaole), ob- 
servation and interaction schedules (again with a 
tendency to develop original scales) and an increas- 
ing number follow a descriptive, case-study ap- 
proadi. Dr. Harlen has used an interesting tedinique 
for pulling together those variables whidi are 
amenable to coding. It is a form of cluster analysis 
and she describes it in her evaluation of Science 
5—13, whidi will be published early in 1975. The 



most clearly defined intention of all this work is 
to provide information for decision makers, whe- 
ther they be a project team, classroom teadiers, 
Headteadiers, Local Authorities or whatever. It is 
interesting to note that this attitude to evaluation 
in England existed long before the work of such 
writers as Robert Stake or Parlett and Hamilton 
was well-known in my country. Equally, of course, 
the Research Team of the Sdiools Council, when 
asked for advice by newly appointed evaluatorS| 
always suggest that they should read as mudi as 
possible about evaluation theory from all sources. 
It is important that they select from the possibili- 
ties those aspects whidi are suited to their parti- 
cular needs. Evaluation is probably most irrespon- 
sible when those concerned with it allow the appa- 
rent brilliance of one school of thought to dampen 
their own critical faculties. Just as English teadiers 
are encouraged to dioose from the curricular op- 
portunities whidi are available, so evaluators are 
urged to dioose from a wide range of possible 
evaluation methodologies. 



Hea^ofis for the beginning of a change in council 
policy towards self-generated, schooUhased projects 

About a year ago, the main policy Committee of 
the Sdiool Council decided to begin to explore ways 
in whidi local curriculum development might be 
both stimulated and helped more directly. No doubt 
there were many reasons for this proposed diange 
of policy and no formal statement of motivation 
was made. Howewer, it seems to me that there 
were two strong reasons, one partly negative, the 
other strikingly positive. 

The negative reason is that we, in common, I 
understand, with most other countries whidi have 
attempted curriculum change, have become some- 
what disillusioned with the R, D and D model. The 
diief problem arises from the final "D** — "diffu- 
sion**. The plain fact is that, no matter how good a 
curriculum development project might be, disse- 
mination, diffusion and take-up have tended to be 
disappointing. Equally, if we think of the ^'Centre- 
Periphery** model, the work at the centre might 
have been good, but the spread outwards to the 
periphery has not always taken place. It was for 
this reasoa that the Sdiools Council set up a 
Working Party on Dissemination, whidi has recently 
produced a report and recommendations for action. 
We have to remember that there are many forces 
working against curriculum development. In Eng- 
land, teadiers have been thrown into tremendous 
dianges in sdiool organisation at the very moment 
in time when bodies sudi as the Sdiools Council 
were urging dianges in the teadiing and learning 
process. Many teadiers have been so preoccupied 
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with the diange to comprehensive schools that they 
could reasonably claim that other dianges are 
inappropriate or even impossible at this moment 
Similarly, the rapid nature of diange in Society 
has imposed heavy burdens on teadiers, many of 
whom are so preoccupied with their day-to<^ay 
problems that they have little or no time or inclina* 
tion to take on the additional demands whidi 
curriculum development is likely to make of them. 
There is a paradox in this situation in that mudi of 
the development whidi they might otherwise take 
(ip is aimed at helping them to solve some at least 
of their immediate problems. Something clearly 
has to he done about this. One possible solution is 
to put some effort into looking more closely at the 
•*Poriphery-Periphery*' model. The theory behind 
this is that teadiers might be more amenable to 
precept than to preadiing. If they can see at first 
handy perhaps in their own schook, perhaps in 
those of Teacher Centre colleagues, the advantages 
which could accrue to them from participation in 
curriculum development might become more appa- 
rent. There are, of course, many difficulties in this 
"P-P" model and it would be most unwise to put it 
forward as a panacea, but the fact remains that 
there is a place for more initiative at the periphery 
and somewhat less at the centre. 

The positive reason is simply that although there 
is a tendency to think of curriculum c evelopment 
as a feature of the 1960s, it has, in /act, been a 
continuing process over time. There nave always 
been teachers who have tried out new strategies, 
new methods of teaching and who hiive produced 
their own materials of various kinds. It can be 
argued that all that is new is the elevation of 
curriculum development to a more ovirtly impor- 
tant and sponsored position. This is why many of 
the best projects have been largely the work of 
successful practising teachers. However, nobody 
would pretend that anything like the total poten- 
tial resources have been tapped. If ways can be 
found oi locating the best work which is going on, 
stimulating it by practicable means and making 
it^ contribution more widely known and more 
generally available there could well be a new 
stimulus to curriculum ciiange and one whidi, 
because it is firmly rooted in the reality of teadier 
needs, could be more effective than the traditional 
C - P model. 

Some examples of local projects which are being 
considered for support 

When this intention of the Sdiocls Council became 
known there was an immediate response. Many 
groups of teachers, together with College of Edu- 
cation and University staffs began to seek support 
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for local projects whidi were already in existence 
but at varying stages of development. I will give 
a few examples by way of illustration. One of our 
Local Authorities has been working for 5 years on 
a Mathematics project whidi aims at providing 
support, guidance and teadier and pupil material. 
The demand for this help has been so strong that 
the Project Team have had no opportunity 
properly to evaluate what has already been done. 
They have therefore asked for support for a much- 
needed evaluation programme. A group of teachers 
in another area has been looking at particular 
points of difficulty met by pupils in Mathematics. 
They have asked for help in this work and its 
extension to methods by whidi teadiers deal with 
the situation. Two Colleges of Education and one 
University Department have been working with 
local teadier groups on Social Science projects and 
they too have asked for support. Similarly a 
number of teadier groups have been concerning 
themselves with the problems of mixed ability 
teadiing, and they too are .seeking support One of 
the most interesting of these local sdiemes is one 
whidi comes from the existing work of a group of 
teadiers at a Teadiers' Centre. They are concerned 
with problems of communication, particularly oral 
English. It is interesting to note in this connection 
that the teadiers concerned have made themselves 
familiar with recent research into communication. 
Perhaps the gap between researchers and teachers 
is not as wide as some people have suggested. 

The point whidi is common to all these requests is 
that they are all, in some degree, local equivalents 
of national projects. In so far as this true it will be 
clear to you that evaluation strategies will have 
much in common with current practice in national 
projects. However, there are two reservations; it 
will clearly be mudi more difficult and dangerous 
to generalise from the ^^local" than from the 
''national"; evaluators will also have to be con- 
tinuously conscious of the fact that experience in 
the process of curriculum development is likely to 
be much more important than the product. 

Posstbtltttes for future developments and their 
tmplication.^ for evaluation 

You will have noticed that in the list whidi I have 
just given to you there is nothing from individual 
teadiers. This is natural in that groups already in 
existence obviously start from a position of advan- 
tage. There are many problems in stimulating 
teadiers to seek support for their own curriculum 
development work. At this moment of time the 
Sdiools Council has not evolved procedures for 
locating interesting work, selecting the most 
appropriate and solving the problems of rela- 
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tionships with the Local Authorities concerned. It 
could be that most of the support available will be 
given to groups of teadiers rather than to indivi-^ 
duals. However, if proposals do come in from 
individual teadiers and sdiools, and if they 
attract support, a major problem , and one 
whidi is currently exercising the Schools Council 
Researdi Team, will be how sudi sdiemes 
are to be evaluated. We do not yet have a 
positive answer to this question, but three points 
are fairly clear. The first is that the evaluation 
procedures followed in national projects vill be 
largely inappropriate. It would be impossible to 
attadi a "professional" evaluator in cadi case, and 
great care will have to be taken to ensure that 
evaluation does not in any way inhibit the natural 
development f'^om whidi the work sprang. Secondly 
it is clear to * that in this new situation the 
^valuator's rok will be that of guide, counsellor 
and friend. Thirdly it seems certain that in work 
of this kind the teadier concerned will have to be, 
to some extent at least, his own evaluator. The 
dangers of this will be apparent to you all, but the 



fact remains that in very small sdiooUbased pro* 
jects it is only the teadier who can monitor his 
own efforts, clarify his intentions and estimate his 
level of success. We should remember that good 
teadiers are continuously evaluating both their 
pupils and themselves. They need help if their 
judgements are to be more than merely subjective. 



Conclusions 

I gather from conversations with the people pre- 
sent here that although the contexts in whidi we 
all work are different many of the fundamental 
problems are essentially the same. We have no 
glib or facile answers, but the first problem is 
always adequately to define the problems. One 
important way in which progress can be mado is 
the exdiange of information and experience. Sudi 
an exdiange is facilitated by this conference and I 
am grateful to the Council of Europe and to the 
Federal Republic of Germany for making it pos- 
sible. 



Teachers' behaviour as a variable in evaluating 
school rieform pilot projects 



One of the most formidable problems facing edu- 
cational authorities in their seardi for improvement 
is to get teadiers to accept the reforms they have 
decided upon. We should note and deplore the fact 
that most of the time innovative decisions are made 
with no preliminary estimate of the conditions 
necessary for success or of the inevitable cost of 
adiieving these conditions. 

The responsibility for this state of affairs rests 
most often with over-hasty and voluntaristic 
decision-makers, but it very often rests as well 
with the researdiers and innovators who do not 
pay enough attention to the human factors involved 
in developing the innovation judged desirable or 
indispensable. Most of the time indeed innovations 
are concerned — consciously and principally — 
with curriculum concent, teadiing methods or 
educational structures; and the impression is given 
that these are regarded as the only important 
things in the innovation process. However, in the 
early stage of any innovation a diange in the 
teacher's attitude and behaviour 's often more 
important for the success of an innovation than 
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are dianges in curriculum content, method or 
structure. Even though the Hatorne effect is well 
known, it is rarely taken into consideration in 
decisions to introduce innovation on a general scale. 
This is perhaps because the medianism of this 
effect has not been mastered precisely enough to 
give technical instructions on how to produce it. 

The main aim of my paper will be to highlight a 
methodological problem: how, when an innovation 
is being evaluated, can the teadier's behaviour be 
taken into account so that the educational authori- 
ties can allow for it when taking decisions on the 
introduction of the innovation on a general scale? 



The importance of teachers* behaviour in any 
innovation 

It is now standard practice to distinguish between 
three types of innovations: 

1. Innovation whidi spreads from the centre to the 
periphery 
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A new content, a new method, or a new sdiool 
structure is produced and ^'laboratory'* tested. In 
the light of an evaluation of the results, a decision 
lo develop the innovation is taken and, after 
further evaluation, this is followed by a decision 
to apply the innovation on a general scale. This is 
the standard innovative process in centralised 
countries. Moreover, the intermediary stage is 
often '•forgotten*' for lade of funds and in a desire 
for speed and pseudo*efficiency. This process fails 
quite often, the innovation usually being rejected 
by the teachers or at best deformed, eroded, neutra- 
lised. Consideration of this failure classically 
reveals the importance of the teadier's behaviour 
and of the error mentioned above of not taking his 
bel^^aviour into account in decisions to generalise. 

2v The second type spreads from one peripheral 
point to another 

This is diffuse innovation, produced by teadiers on 
the spot themselves, and spreads from one point to 
the next, somewhat like a fashion. The behaviour 
of teadiers as creators and partisans of an idea 
here is of prime importance. But the process entails 
no evaluation. Either it spreads rapidly often dis- 
appearing as quiddy as it appears, or it isolates 
itself in podcets. 

3. The third type seeks to combine the advantages 
of studied and evaluated innovation with those of 
spontaneous irmovation. With this type of i;mova- 
tion, a dialectic is established between the centre 
and the periphery^ and during this dialectic there 
are phases of routine evaluation. The creative 
initiative belongs to local teams who have decided 
to join together to solve a common problem. The 
centre comes in as a catalyst and instigator more 
than as a creator. Generalisation is linked to the 
progressive creation of curriculum content, method 
or structure; and evaluations intervene as a regu- 
lating force in the creative process. Here too, it 
rapidly becomes apparent that the teadier's 
behaviour is a fundamental factor in the success or 
failure of innovation. Whatever sort of innovation 
is under consideration, though, the job of evalua- 
tion, if it is tc be effective, should be given to a 
team distinct from the teaching teams. The ties 
whidi exist among these teams, however, and also 
the place of teadiers in the evaluation mechanism 
itself are both decisive factors for the validity of 
the evaluation. 

Taking the teadier into account in the evaluation 
of conventional innovation (from the centre to the 
periphery) 

In this case evaluation is carried out at tw» stages 
in the process. First of all at the conception stage. 
In the case of producing teadiing material, evalua- 
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tive checks may be numerous, following eadt 
sequence produced. In the case of structural inno- 
vations, tiiey are less frequent and occur from 
time to time when eadi decision has been followed 
by a period of application (six months, one year). 
At the ideas stage, the teacher's behaviour is very 
rarely taken into account. The important thing for 
the planning team is translating its objectives into 
terms of pupil behaviour (tests, questionnaires). 
The tesi'^er in the sdiool situation is then a member 
of the planning team !./ d is so de facto in a con- 
tinucus training and maximum motivation situa- 
tion. He is rarely the subject of observation. When 
he IS, it is part of keeping a diedc on the operation, 
the e valuator being an integral part of the planning 
team and paying more attention to the pupils at 
grips with the material taught than to the volun- 
teer teacher diosen for the experiment from those 
considered a priori to be good teachers. 

On the other hand there is considerable concern 
with the teacher's behaviour fully, at the develop- 
ment stage (full-scale trial of the innovation 
product). There are two possible attitudes here. 

The more usual, and the easier, is to try to neutra- 
lise this factor through recourse to criteria extra- 
neous to the actual act of teaching. The evaluation 
medianism consists then, in addition to information 
gathered about pupils (general tests, sociological 
data, tests putting the innovators' objectives into 
practice), of information about teadiers carrying 
out the innovations. Thii might include sociological 
files (age, sex, type of training, attitudes, interests) 
to as either to neutralise the teadier factor 
(observation of performances of comparable pupils 
taught by comparable teachers) or to illustrate the 
possible influence on comparable pupils of the 
variables in the teadier files. This might be called 
a macroscopic evaluation of the teadiers' behaviour. 
Such an evaluation method is useful when a rough, 
large-scale judgment of the effectiveness of an 
educational system is needed, but it is clear that it 
does not capture the essential. 

It is noteworthy, indeed, that a closer study of the 
performances of comparable pupils, taught by 
teadiers who seem comparable according to the 
sociological data assembled in the teacher file, 
reveals group phenomena (performance of classes 
taught by a particular teacher, or performances of 
a sdiool taught by a group of teadiers) which 
remain quite unexplained at this level of analysis. 
From the point of view of the effectiveness of an 
innovation this subtler approadi might well b^ thrj 
key one. A closer look i^ould therefore be taken, 
and the reasons for a better or worse performance 
investigated. Teachers' behaviour should be des- 
cribed in an objective and, if possible, quantitative 




manner to equate this behaviour with the other 
variables in the system studied. The various me- 
thods of objectively analysing teadiers* behaviour 
are essential here. They are becoming well known 
through the works of Flanders and, in Frendi, 
those of De Landsheere and Chobeaux. These 
analyses, involving the analysis of features of 
verbal behaviour are valuable for shedding light 
on periodic phenomena. They will nevertheless 
have to be improved to include non-verbal factors 
and fit types of non-frontal teadiing (work groups, 
^'animation'') where are now difficult to handle. 

Two methodological remarks about this type of 
study are appropriate here: 

To begin with, their large-scale use is impossible 
owing to the number of observers necessary, and 
to the need to train them to ensure methodological 
agreement. For full-scale evaluation, the only 
dependable source of information for decisions, it 
is absolutely necessary to work with enough pupils 
to have a representative sample of the national 
sdiool population as regards sex, age, general deve- 
lopment, and socio-professional background. Thus 
the number of classes and teadiers involved should 
itself be fairly large especially if all these teadiers 
are meant to provide a good sample of the teach- 
ing body concerned. To pinpoint the variable which 
teadier behaviour constitutes, a more manageable 
instrument must be used. There is no evading the 
difficulty: only a questionnaire could meet this 
need. But the questionnaire must be based on 
detailed behavioural studies and deal with clearly 
defined forms of behaviour. The major ob&.acle is 
the discrepancy between what the teacher thinks 
he is doing and says he is doing, and what he 
actually is doing. The questionnaire must therefore 
deal with facts and not opinions (eg: ''How many 
word5 did you make your pupils learn by heart 
during the week beginning . . .?", and not "Do you 
make your pupils learn texts by heart often, some- 
times, not at all?"). It must also be filled in by an 
investigator and supported by proof. 

The second danger is that teadiers' behaviour is 
liable to be studied in the light of certain standards. 
It is the natural temptation when the tested pro- 
duct has been previously developed in the labora- 
tory and is now being "sold" with instructions for 
use. There is good and bad use of material. This is 
the case with pupil tests who have been evolved in 
the laboratory and become a standard of reference 
for the evaluator with evaluation in the form of 
measurement of deviation from the norm. The 
failure of the pupils can then be ascribed just as 
easily to the behaviour of the teadier as to his mis- 
interpretation (from the evaluator's point of view) 
of the objectives implicit in the products used. 



It is tempting in these circumstances to blame 
failure on inadequate training, whidi may be the 
case. But it may also be due to the basic inade- 
quacy of the new material in real sdiool situations, 
resulting from an insufficient or non-representative 
number of teadiers and pupils having been involved 
in planning the innovation. An imposed objective 
is generally doomed unless account has been taken, 
from the beginning, of the real state of the educa- 
tional system. New curriculum content in mathe- 
matics, for example, needs to have been created 
from the beginning, by involving all and sundry, 
teadiers and pupils, in the process, if the result is 
not to be a merciless instrument of social selection, 
as has often happened with this subject. 

This is why, when an innovation has come from 
the central authority it appears advisable to observe 
the educational system and teadiers' behaviour 
just as they are, without any preconceived ideas, 
an attempt being made to v/ork out objective 
typologies related to pupils' performances, whidi 
should themselves have been observed without 
referoice to any pre-determined standards. Sudi 
action may certainly hold surprises for the central 
policy-makers on innovation. 



Taking the teacher into account in the evaluation 
of supervised innovation (from one peripheral point 
to another via the central) 

In the preceding descriptions I have taken for 
granted the co-operation of teadiers, the subjects 
of evaluation, and this is an aspect that must not 
be overlooked. Evaluation at the development 
stage of a downwards innovation always encoun- 
ters a more or less clearly-stated refusal by 
teadiers to be judged from the outside as opera- 
tives and subjects for study. Teachers habitually 
diallenge the validity of pupil-tests in sudi 
operations. This not only reflects emotional reac- 
tions, but often points out differences between the 
objectives of the "policy-makers'* and those of the 
experimenters. In the same manner the evaluator's 
surreptitious recourse to the norm is always seen 
by the teadier as a personal attack. 

It is clear that the evaluation of spontaneous 
innovations would be viewed as being even more 
oppressive. 

This being the case, innovators almost always 
reject outside evaluators as being inadequate. When 
evaluators are imposed upon them, the teachers 
receive them distrustfully and challenge their 
methods and findings, calling them irrelevant to 
the objectives sought. This is frequently the case, 
moreover, as these spontaneous innovations usually 
result from ideological and ethical considerations 
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that are very difficult to express in any final, 
observable behaviour, if only because of the long* 
term effect assumed by the innovator. But it is too 
convenient for the innovator scornfully to reject 
anything he cannot catch in the nets of an objective 
methodology as "pipe-dreams". 

Thus, the evaluation of a spontaneous innovation 
has no reafdiance of success or effectiveness unless 
it is not only accepted by the innovators but is 
actually requested by them as a means of facilitat- 
ing and regulating the innovation. Terminal (or 
summative) evaluation can only result from a 
succession of partial, regulatory evaluations in 
which the teachers-innovators have an important 
role to play. 

There are two cases where the conditions necessary 
for sudi acceptance and participation may exist. 
Either, the innovators themselves will ask for the 
support and criticism of an evaluation team. This 
presupposes mudi mutual confidence and usually 
a lowering of the initial level of enthusiasm found 
with any new spontaneous innovation. There must 
be at least concern or suspicion about the real 
effectiveness of the action undertaken. 

Or, the idea will come from the centre, albeit in a 
general context favourable to the idea, whidi is 
merely officialised by the centre. In this case 
periphery needs to be appealed to for a voluntary 
commitment to an innovative action, in whidi eadi 
of the participants has from the beginning an equal 
voice in the discussions and experiments, and con- 
sensus is the result of continuous product compari- 
son and evaluation. 

Such a process enables the distortions and bottle- 
necks of. conventional innovation processes pre- 
viously described (from the centre to the periphery) 
to be avoided. But several conditions are necessary: 

1. The teadiing teams must be involved in the 
designing of the innovation from the very beginn- 
ing. 

2. The pupils and teachers taking part must form 
a representative sample of all those who would be 
affected by the introduction of the innovations on 
a general scale. In other words, the teachers in- 
volved should hot be hand-pidced for their merits 
or ability; their readiness to participate should be 
the only consideration. 

3. The teadiing teams should be organised in the 
form of sdiool units, eadi of whidi should be taken 
diarge of locally by a design and evaluation team, 
organically and democratically linked with the 
co-ordinating central authority. 

4. The various teams thus organised should be 
involved not only in the innovation itself (method, 
structure), through local trials and periodic com- 



parisons of results, but also in the progressive 
evaluation of the innovation, the final instruments 
being a synthesis of the partial instruments devised 
by them as part of the innovation. 

5. The teadiers should themselves be involved in 
the conduct, marking and interpretation of pupils* 
tests. 

.\s the researdi will immediately become full scale, 
the teadicis' behaviour will accordingly have to be 
taken into account from the beginning of the 
innovation and not only during the development 
phase. There will have to be investigations, from 
the beginning, into how the teams organise or fail 
to organise themselves, how they interpret the 
general initial objectives and the theoretical infor- 
mation given, how th^ apply them, the positive or 
negative role of the group leaders, the scale and 
nature of co-ordination, the possible influence of 
local material conditions. Only if this is done can 
an informed decision to introduce an innovation 
on a general ^^ale be taken by the education 
authorities as tK general application of an inno- 
vation must recreate the innovation processes a^ 
training processes for the teadiers. 

Teadier observation tediniques will be those 
previously indicated for conventional innovations, 
namely, questionnaires based on and verified by 
firsthand observations. 

Clearly, the cost is much higher than with conven- 
tional operations — too high, it is tempting to 
think. 

It seems to me, however, that the following con- 
siderations make the process not only desirable, 
but indispensable to anyone who really wants to 
diange the educational system and not just its 
statutory appearance. 

1. Sudi a process can only be engaged in when 
there is an adequate political consensus on the 
need for diange. The failure of a structure, inade- 
quate pupil performance, a diange in the way a 
subject is studied at university, all provide sudi 
occasions. 

2. Full-scale innovation must cover — from the 
beginning and on a voluntary basis — the largest 
possible number of training centres since a super- 
vised innovation process is already one of continuing 
education. It prepares innovating centres to play 
locally, in the phase of general application, the 
role of encouragement and co-ordination played by 
the national centre during the research phase. 

3. Sudi a process, if it is to stem from a national 
consensus on the problem posed and from the need 
for a solution, must in no way be the product of 
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preconceived notions regarding the natuie of the 
solutions to be applied. These solutions must 
emerge progressively from encounters in which the 
various parties concerned (teachers, parents, pupils, 
local representatives) participate throughout. In 



such a process of change, observation of teachers* 
behaviour becomes a dialectical moment, and, at 
the same time, a means of becoming scientifically 
aware of the conditions necessary for the success 
of a generally applied innovation. 



Evaluation as a system component of educational development 



It has often been observed that there are gross 
contrasts between the high pretensions of attempts 
at innovation in the area of education and the 
modest results obtained. The reasons for these 
contrasts most likely lie in an under-estimation of 
the complexity of the problems, as well as in the 
rush towards and pressure for spectacular solu- 
tions. Such solutions are often more influenced by 
the interplay of special interests as well as local 
determinants leading to narrow views of the matter 
at hand rather than by an objective and com- 
prehensive research approach. 



How; can educational development be made more 
eiiectivc? 

Empirical educational research is still in the early 
stages of its development. On the one hand, the 
manner of working and the expectations are, in 
many respects, still under the influences of the 
speculative pedagogical tradition which often 
explains things very simply. On the other hand, 
its methods have been borrowed from the natural 
sciences and technology, which deal with more 
easily defined, clearer and less complex structures. 
Therefore, it may be assumed that the hitherto 
unsatisfactory innovational techniques — which, in 
many cases, are based on an oversimplification of 
the problems involved — could be developed to a 
greater degree of efficacy if one were to systemati- 
cally accentuate and show regard for the complex 
character of educational matters. 

Everyone involved in this task needs to go through 
a new learning process: pupils, parents, teachers, 
administrators, various interest groups, politicians, 
researchers and experts in the field. In order to be 
able to cope with such a difficult learning task, 
special methods and organisational structures are 
necessary. 

The mcthodolofifi/ involved in the initiation and 
permanent establishment of an efficient process of 
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innovation is a kind of systems analysis. This would 
lead to more intensive consideration of evaluation 
standpoints as related to all phases and components 
of the developmental projects. 

In the organisation of the developmental activities, 
it is necessary that there be systematically co- 
ordinated co-operation between practical expe- 
rience, development and research within the 
framework of appropriately selected key projects 
on a national and international basis. This necessity 
follows from the systems analysis approach to the 
problem. 

During the last few years, the meagreness of the 
results of numerous innovations has become in- 
creasingly apparent as a consequence of technically 
better and more intensive evaluative endeavours. 
In reaction to this, many new undertakings are in 
the making for co-operative, comprehensively 
evaluated developmental activities. These are also 
a by-product of more dynamic educational policies. 
It is important now to further develop and to 
broaden these early attempts. 



A systems analysis approach 

Systems analysis, as used here in this connection, 
appears at first glance to be an extremely trivial 
heuristic strategy to solve complex problems: real 
or conceptual facts are considered components of a 
comprehensive system within which these facts are 
interrelated in a particular way. Elements of a 
system can again be looked upon as systems them- 
selves; and systems can be examined as components 
of more comprehensive systems. 

In the treatment of problems by means of systems 
analysis one aims at determining, as completely as 
possible, the systems relevant to the specific 
problem and their internal and external rela- 
tionships. Whenever the solution of a problem is 
facilitated by mapping procedures, then graphs or 
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mathematical models are constructed, or classifi- 
cation sdiemes for ordering goals, actual facts, 
working concepts, alternative hypotheses, open 
questions, etc are worked out. In this way, the 
systems analysis used initially under rather general 
aspects gradually yields to techniques of analysis 
specific to the particular field examined. 

From developmental projects planned and directed 
by a systems analysis methodology better results 
can obviously be expected than from an improvised 
procedure considering a smaller number of relevant 
factors in a less systematic manner. 

Various problems arising in this connection can 
only be briefly touched upon here. Above all, 
responses are sought to the following questions: 

— To what extent is the systems analysis method a 
leamable heuristic technique? 

— To what extent can it be formalised and deve- 
loped? 

— To what extent is thinking in terms of systems 
analysis a normal consequence of research and 
development in the various scientific and techni- 
cal disciplines? 

— To what extent is it based on relatively stable 
characteristics of personality and intelligence 
that cannot be easily influenced? 

— How efficient is the systems analysis method 
with respect to various types of projects? 

— What forms of negative side effects could appear 
in connection with the systems analysis method? 
One could imagine, for example, that the treat- 
ment of the problems at hand would not always 
be aided hy extensive use of graphs or mathe-^ 
matical models or by classification systems. 
Models and diagrams can possibly impede or 
fixate thinking. 

A first step towards the practical consideration of 
aspects of systems analysis would be the com- 
prehensive evaluation of developmental projects. 
This evaluation would not only relate to the desired 
results, but would necessarily need to accompany 
and control all the components of the conceived 
project, viewed as a system: setting of goals, 
planning, working out, implementation, and inten- 
tional and unintentional effects. 

Such a comprehensive evaluation should bring 
developmental projects into an integral relationship 
with the systems of educational goal setting, the 
fundamental findings in psychology and sociology, 
teaching methods and instructional technology, as 
well as and in particular*, with learning and teach- 
ing as practiced. 



Reform of project organisation strategies 

An organisational reform of research and develop- 
mental activities by means of an extensive partici- 
pation 0/ teachers on the job, would make it 
possible to realise not only greater democratisation, 
but also: 

— the necessary intensification of action researdi; 

— the development of efficient implementation 
techniques; 

— the gaining of teachers, who have proven their 
abilities in developmental work, as qualified 
staff members for the centres; 

— and a theoretically well-grounded and at the 
same time action-related system of teacher 
training. 



Examples 

Two types of project seem particularly suitable for 
preparing the way for a co-operative developmental 
system which appropriately considers the evalua- 
tion standpoint. These projects are: 

• experiments in changing the school structure; 

• projects related to providing for continuous 
feedback of learning achievement to pupils and 
teachers as an essential diagnostic, motivating 
and ini^tructional function. 

Two projects have been selected from the program- 
me of Austrian school experiments to illustrate 
these points: 

The structure-changing project ''Experiments in the 
school for the ten to fourteen year olds" is mainly 
concerned with the development and evaluation of 
a form of comprehensije school in which mother 
tongue, English and mathematics are taught in 
achievement levels (setting). All other subjects are 
taught in heterogeneous classes. A political con- 
sensus could not be reached on the introduction of 
a comprehensive school system. The political par- 
ties, however, did agree to a law which prescribed 
evaluative experiments along the^e lines in order 
to gain information necessary for further decisions. 
Several details of this project are contained in' 
the background paper. Part I. In thr project under 
consideration, the following points should be 
stressed: 

— The comprehensive school project raised among 
politicians, administrators and teachers a marked 
demand for evaluation and initiated a learning 
process related to an understanding of the 
principles of evaluation and the possibilities it 
offers. 
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^ An institutionalised co-operation was established 
between university research, sdiool administra- 
tion, and the Centre for Sdiool Experiments 
within the framework of a scientific advisory 
board for the evaluation of sdiool experiments. 

— In connection with the teadiing of syndironised 
curricula for three different levels in the 
respective adiievement groups, many teadiers 
developed a pronounced problem-consciousness 
in the selection and structuring of learning 
objectives as well as in assessment procedures. 
Teadier groups work together with educational 
researdiers and experts in attempts at setting 
up curricula. 

— Forms of instruction are being set up whidi 
systematically try to take into consideration 
differences in the pupils' learning behaviour: 
learning objectives are being adopted to the 
needs of the various adiievement groups and 
individualised learning materials are being 
developed. 

— The problems of the low adiiever, who comes 
diiefly from the lower social levels, became 
clearly visible to the teadier. The teadiers 
recognise that these pupils require special help 
so that they are not continually frustrated by 
failure. This insight emerges in particular as a 
result of more exact measurements of per!r)r- 
mance in the experimental sd:ools; as well as 
in connection with the introduction of English 
instruction whidi seems to be very difficult for 
this group of pupils. 

— Finally, the comprehensive sdiool project has 
succeeded in triggering off a lively and con- 
tinuing discussion between politicians, represen- 
tatives of interest groups, teadiers, administra- 
tors and sdiolars, on educational problems, 
which are based partly on provisional results of 
evaluation. 

The second project to be described here relates to 
formative adiievement testing and the optimisation 
of psychological learning conditions. It carries the 
working title **Learning Test Project". At present 
this project is in the stage of development of 
materials. Part II of the background paper contains 
information about several of its fundamental prin- 
ciples. 

The learning tests offer prompt and detailed 
feedbadc of the learning results to the student and 
to the teadier. On the basis of a simple presentation 
of data, insight into long-term learning develop- 
ment can also be gained. Learning tests differ from 
the usual formative tests in that they do not fuli:I 
only diagnostic functions, but, on the basis of their 
specific structure and application, also serve as 



learning material. When the pupil goes through the 
various test items, he goes through the process of 
problem-solving, principle-learning, concept-forma- 
tion, discrimination-learning, memorisation, auto- 
matisation, etc. 

The procedure necessary for using a system of 
learning tests is not as simple and readily 
understandable, as is teadiing someone how to 
operate a TV set. It should be compared instead 
with learning to play a musical instrument or with 
running a computer, whidi have to be studied as a 
special art. Therefore, the effective handling of 
learning tests must be developed in pioneer experi- 
ments involving the intensive co-operation of both 
teadiers and pupils. In subsequent implementation 
experiments it must be made teadiable by routine 
training. 

The development of learning tests is a long-term 
task. In interaction between instructional practice 
and corresponding researdi in educational psydio- 
logy, the test materials must be continuously 
improved on the basis of formative eva*iation, 
simultaneously with the further improvement of 
methods of application. Thus, the learning tests 
always need to be integrated with the other com- 
ponents of the curriculum, also under continual 
development Included in these are the teadier's 
lectures, class discussions, work in small groups, 
games, co-operative projects, programmed instruc- 
tion, etc. 

In contrast to the traditional instructional materials, 
learning tests will not be available within the 
foreseeable future as ''finished products". Of 
course this is also true of other curricular materials 
now under systematic development. However, one 
will not wait to turn them over for use until they 
are "completed". Rather, they will be brought out 
in a relatively early stage of development, as soon 
as they can be viewed as making a contribution to 
an improvement in instruction. By far the more 
comprehensive part of the developmental work 
should be carried out on the basis of continually 
developing evaluation in broader and more syste- 
matic co-operation between actual performance on 
the job and researdi, and in continual and complete 
integration with other curricular projects. 

By means of learning tests, the teadier is given the 
opportunity to diedc the effects of his instruction. 
In this way, he is directly confronted with educa- 
tional problems relevant to him personally. This 
continuous confrontation with actual problems 
encourages the teadier to come to grips in a 
productive manner with an action-oriented theory 
of instruction and with ways for improving his 
educational tediniques. 
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It should be possible to offer co-operating teachers 
the stimulus and the opportunity of acquiring 
scientific knowledge and skills relevant to the 
re^ective projects in educational psydiology, 
evaluation methodology, curriculum theory, statis- 
tics, etc. On the basis of such knowledge some 
teachers might acquire further qualifications for 
researdi and developmental activities. 

Those teadiers, too, who only utilise the system of 
learning tests without co-operating in its develop- 
ment should be given the opportunity to acquire 
practice-oriented scientific knowledge. 

Since the pupils can immediately diedc the results 
of their learning, they are in a position to evaluate 
and control their own learning behaviour. Oppor- 
tunities for a partnership-like co-operation and 
discussion between pupils and teachers arise when 
the methods of handling the test efficiently and 
evaluating its results are being studied. 

It is the responsibility of the Development Centre, 
among other things: 

— to develop, or to give advice on, the first draft 
of a learning test system and to prepare pilot 
projects; 

— to evaluate modifications or extensions of 
materials and application tediniques worked out 
in the individual learning groups, to integrate 
them into the systt?^., and to make them 
available to all persons concerned; 

— to deal with problems of organising the learning 
process on the basis of statistical effect analysis, 
theoretical reflections and special studies. 

In all these tasks which, to a considerable extent, 
are of an evaluative kind, the centre has to co- 
operate both with the people engaged in practical 
instruction and with those doing fundamental 
research. 

The division of labour between periphery and 
centre is to be more systematically developed on 
the basis of evaluative information gathered in the 
course of project work: on the one hand restrictive 
tutelage of teachers has to be avoided; on the other 
hand provisions must be made for securing appro- 
priate horizontal and vertical co-ordination as well 
as scientifically and technically satisfactory project 
work at the periphery. 

Formative evaluation within the learning test 
project will be based on two feedbadc systems. 

Feedbadc System No. 1 (FS 1) yields immediate 
information eg by simple display of item difficul- 
ties at various achievement levels within individual 
tests or across sequences of tests. More or less 
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systematically gathered observations, proposals and 
reactions of pupils, parents and teadiers constitute 
further information to be gained by FS 1. This 
feedbadc system enables the project workers 
roughly to adapt their constructional techniques, 
principles and assumptions without delay. In many 
cases sudi approximate corrections can lead to 
remarkable improvements. Over and above this, 
Feedback System No. 1 stimulates and supports 
those learning processes in teachers and pupils 
whidi enable them to participate productively in 
action researdi. 

Feedbadc System No. 2 works at a more sophisti- 
cated analytic level. It deals primarily with those 
problems relevant to classroom learning and 
teadiing whidi reach to some extent into the field 
of fundamental research. 



Structuring a developmental system 

As is shown by the example of the learning test 
project conceived along the lines of systems ana- 
lysis, the functions of evaluation, theoretically 
controlled research work, developmental activities, 
implementation and teacher training are closely 
interrelated. Comprehensix^e project evaluation, as 
a test of: 

— project goals, 

— theoretical foundation and technical suitability 
of planning, 

— practical realisation m instruction, 

— and intentional and unintentional effects, 
provides for a continuous and minimally-biased 
outlook on the whole problem situation and contri- 
butes, in doing so, to the integration of the various 
fields of work participating in the project. 

The more one succeeds in integrating the conceived 
goals of both experienced teachers and researchers, 
the more productively a co-operative developmen- 
tal system will function. This should be adiieved 
insofar as the researchers work together with 
teachers on the job, using formative evaluation, on 
the problems arising out of daily learning and 
teaching processes, and they then relate their 
research goals to the outcomes of these evaluations. 
And the teadiers should develop their interests in 
educational reform in communication with re- 
searchers in a continual confrontation with reality 
on the basis of as comprehensive an evaluation of 
their classroom instruction as possible. In this way, 
they could emancipate themselves from hampering 
tradition as well as from one-sided trendy ideas. 

One might expect — without being too Utopian — 
that a more efficient educational research and 
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developmental system could be worked out. The 
speed with which this all could happen would more 
or less depend upon how favourable were the sur- 
rounding circumstances. 

Such a developmental system needs to be based 
upon European or international co-operation be- 
cause of its complexity and the enormous extent of 
the problem. The working out of means and 
methods for co-ordinating projects on a European 
basis has already been tackled and shows great 
promise. The Council for Cultural Co-operation has 
made a substantial contribution towards the 
establishment of a starting point for co-operative 
work on projects through their . European trend 
reports, the planned contact projects workshops, 
and the information system EUDISED. 

These beginnings call for supplementation by 
means of a scientifically founded strategy for the 
optimal choice of project goals. Such a strategy 
would need to be based on a catalogue of the 
identifiable current innovation problems in educa- 
tion in the various countries. Naturally, the plann- 
ing of the projects will often be decided not 
according to scientific principles, but on a political 
level. Nevertheless, the political decisions can be 
adjusted and influenced by scientific criteria of 
selection worked out on a reliable basis. 

The mode of operation and the horizon of the 
aspired developmental systems would have to differ 
essentially from the traditional manner of educa- 
tional research and developmental activities. Until 
now, one dealt very often with relatively unco- 
ordinated and short-sighted individual projects 
which were "terminated" with a more or less 
comprehensive scientific publication or with the 
publication of "finished" leam:ng materials. The 
perspective of such projects is usually rather 
narrow, that is, various components of a system 
are placed in the foreground, and other no less 
relevant components are not given sufficient con- 
sideration. The great complexity and multitude of 
aspects in educational problems, which are fre- 
quently also burdened by ideological considerations, 
appear to be difficult to master without special 
systems analysis techniques. 

A modern research and developmental system 
should begin with relatively few very carefully 
chosen key projects which can bring about a system 
of complementary project associations. The way to 
such project associations could be opened up in an 
especially effective manner through the consistent 
application of the principle of comprehensive eva- 
luation. 

N'^w problen.s appear repeatedly as a result oi ich 
evaluations- These can generate subsidiary projects 
within the areas of implementation, basic research, 



methodology and goal determination. To bring 
about the necessary contacts with eligible project 
workers in the respective areas, a well functioning 
communication system and appropriate organisatio- 
nal provisions must be established. 

Several aspects of the establishment of an integrat- 
ed developmental system can be illustrated by 
means of the example of the two Austrian projects, 
already mentioned. 

Selection criteria jor key projects 

An analysis of the innovative relevance of these 
projects yields two main types of criteria for the 
selection of key projects: 

First, criteria whidi relate to the suitability of a 
projects jor the mobilisation oj innovative interest 
and for initiating learning processes within the 
important groups of persons involved in educatio- 
nal innovation. One would need to examine in 
particular how one could best awaken the interest 
of parents, teachers, scientists, administrators, poli- 
ticians and the general public, and which learning 
goals are primarily to be attained by these various 
groups. 

The example of the Austrian comprehensive sdiool 
experiments leads to the assumption that projects 
which change the organisational structure — in 
contrast to attempts at changes in teaching methods 
for instance — arouse especially great and very 
widespread interest right from the beginning and 
also gradually lead to a consideration of a variety 
of educational problems. 

A second type of selection criterion for key pro- 
jects relates to the suitability of a project as a core 
for the establishment of a project association. 

Core projects presumably will have to have goals 
with far-reaching significance and be directly 
applicable to the school setting. They should be 
able to be organised according to principles of 
action research and require permanent co-operation 
between practicing teachers and researdiers of 
various disciplines. 

A comprehensive formative evaluation to examine 
all components of the project should continually 
lead to the identification of essential problems 
which then can be dealt with in subsidiary pro- 
jects. 

Core projects are expected to lie mostly in the 
field of curriculum development. Here teachers and 
researchers can productively meet one another in 
the realistic setting of classroom learning and 
teaching, whence the problems branch out. 
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The evolution of a project a$ioeiation 

The Austrian learning test project can be viewed 
as an attempt at the establishment of a core pro- 
ject It should consist of co-ordinated partial projects 
carried out by separate project teams who develop 
learning test systems as components of curricula 
for various school subjects. Developmental work 
has already begun for mathematics and English 
as a foreign language. 

Taking the projects for mathematical learning 
tests as an example, I can only allude to the way 
in whidi subsidiary projects are beginning to be- 
come linked to the core project. These were to be 
worked on by specialised project teams in com- 
munication with eadi other and with the core team. 

Pupils with various aptitudes demonstrate great 
differences in speed of learning. These differences 
may vary in size with respect to various types of 
learning goals: e.g. low adiieving pupils appear to 
be able to adiieve goals in mattiematics at the 
lowest level of the Bloom Taxonomy relatively 
easily. At the same time, they require a great deal 
of time for certain other types of mathematical 
goals which generally lie at higher levels. The dif- 
ferences in learning ability give rise to various 
problems in the learning of mathematics. Of these, 
only two will be pointed out here: 

The first problem concerns the sequencing of the 
learning steps for pupils at various levels. In this 
case, the following questions arise: 

— Are there different optimal learning paths for 
readying certain learning goals in mathematics 
for pupils of varied aptitude levels? Or does the 
proportion of their learning time remain con- 
stant even when the learning path is varied? 

— If there are optimal learning paths for specific 
aptitudes, which characteristics of the learning 
path or of the individual learning steps can 
serve as criteria for the optimisation of the 
learning path for pupils of various aptitude 
levels? 

The second problem area connected with the diffe- 
rentiated levels of the pupils relates to the 
question: how should the final goals for mathema- 
tical instruction be reconciled with the various 
aptitude levels? For example, when the learning 
time differences between the levels for various 
types of learning goals surpass a certain standard, 
then the acute question arises: how mudi learning 
time should be spent on the various learning goals 
at the individual aptitude levels, in order that the 
optimum set of goals can be adiieved at eadi 
level? 



Further problems arising out of the prospective 
core project *^Leaming tests for mathematics** can 
only be briefly mentioned here: 

— The development of tedmiques for the analysis 
of the learning process based on learning test 
results. Above all, at issue is the working out 
of criteria for the optimal sequencing of the 
learning steps. 

— An analysis of the relationship between the lo- 
gical and the psychological structure of mathe- 
matics with special emphasis on a critical ex- 
amination of the significance of the so-called 
"New Math** for the learning of mathematics. 

— Working out criteria in order to determine final * 
aims for mathematical instruction. 

— Integration of the learning tests with the re- 
maining components of the curriculum, particu- 
larly with the co-operative and autonomous 
forms of learning. 

— Integration of the learning of mathematics with 
learning in other aveas. 

This list of researdi and developmental problems 
reveals only a small selection of the multitude of 
problems whidi have already become apparent. 
Scarcely one of them has a practically applicable 
scientific solution. For that reason these problems 
had to be treated in corresponding subsidiary pro- 
jects. 

It does not follow however, that the core project 
has to wait for its developmental work until the 
subsidiary projects or other researdi have brought 
about usable results. On the contrary, the core 
project produces results on the basis of the best 
information available at that particular time and 
uses the results of subsidiary projects and other 
researdi as soon as they are available. 



Summary 

This attempt at sketdiing an integrated develop- 
mental system intends no more than to give an 
initial estimate of a course to be taken. The tedi- 
niques for the planning and organisation of sudi 
a developmental system require comprehensive 
evaluation in order to bring about continual im- 
provement and to accomodate to current scientific 
and social developments. 

We will need to adjust ourselves to the fact that 
spectacular results can seldom be adiieved in short 
spurts in the area of education. Gradual but sure 
progress can be adiieved on a broad front if one 
makes far-sighted plans, and then, on the basis of 
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patience and adequate resources, builds up a co- 
operative developmental system controlled by com- 
prehensive evaluation. 

The educational system now under development 
can be viewed as a highly complex cybernetic sy- 
stem with many subsystems to be regulated. It is 
not only a question of continuous corrections of 
deviations of actual values of the system function 
frori desired values by resources inherent in the 



system, but also of controlled development of the 
components of the systems themselves: the sensors, 
the feedbadc medianlsm, the executing body, and 
the method of analysis for the desired value. 

It is the job of a comprehensive evaluation as a 
systems component of developmental projects to 
continually insure the information basis for sudi 
an extensive, co-ordinated, controlled develop- 
ment. 



Summing-up lecture 

by Professor W. MITTER, 

Deutsches Institut fiir Internationale Padagogisdie 
Forsdiung, Frankfurt/Main 



In all the countries whose experience we are dis- 
cussing here, the scientific evaluation of pilot pro- 
jects for sdiool reform is in the initial experimental 
stage. This has been emphasised and illustrated by 
the lectures and participants at various points. In 
their statements on this experimental stage the 
speakers have referred to the devising and testing 
of appropriate methods and of the instruments 
whereby they are applied as well as to the esta- 
blishment of organisational models on whidi to 
base the evaluative research projects. 

We have only to bear this in mind to realise that 
the symposium could not be expected to furnish 
ready-made solutions, mudi less practical directi- 
ves. 

It seems to me, on the other hand, that we were 
justified in expecting that on the basis of their 
accounts of the theoretical researdi or practical 
experiments carried out by them the lecturers 
would develop ideas whidi would be of use for the 
further development of theory and the practical 
evaluation of pilot projects for sdiool reform. The 
justification for this is to be found in the discus- 
sion groups, each of whidi took up in its own way 
the ideas put forward by the lecturers and devel- 
oped them further by integration and contrast in 
equal degrees. 

It is not surprising that all group discussions di- 
. verged repeatedly from the main theme and ex- 
tended to general problems of educational innova- 
tion and decision-making processes, seeing the in- 
terrelations between these questions and evalua- 
tive researdi. 

These interrelations must also be apparent from the 
group of basic questions on whidi I should again 




like to dwell now, in winding up. It is not my 
Intention to assess the meeting but to summarise 
my own direct impressions as a participant and a 
learner. 

I will be very brief and confine myself to three 
groups of questions whidi seem to me to create 
dialectical tensions — dialectical, in that the phe- 
nomena involved may be considered as intitheses 
whidi pervade educational, scientific and social 
processes, neutralising eadi other and thus giving 
rise to new tensions. 

I refer in the first place to the conflict between 
centralised and decentralised innovation in sdiool 
reform. Most of the lecturers came out in favour of 
school-based, decentralised educational development 
and emphasised its functional advantages as well as 
its emancipatory ones to whidi I shall revert. Not 
only Mr. Voigt, as a representative of sdiool ad- 
ministration in a position of political responsibility 
but the other speakers too dwelt on the need for 
central overall direction of sdiool innovation, 
though sometimes in a modified form. This led — 
I believe rightly — to doubts being expressed 
about the existence of a general trend towards de- 
centralised innovation, and the statement on the 
subject by the Minister of Education of Land Hesse 
might be regarded as virtually the "official evi- 
dence". 

It would surely be an over-simplification, and tant- 
amount to devitalisation, to regard direction of 
central national or international, as it will no doubt 
increasingly become — education policy by state 
or public bodies merely as a necessary evil or a 
disturbing factor. That would mean radically over- 
estimating the capacity of decentralised model 
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sdiools to prepare pupils for a share in solving ge* 
neral social problems. 

The second conflict is between the two different 
kinds of researdi repeatedly referred to here: ac- 
tion research as an ally of decentralised sdiool de- 
velopment and traditional empirical researdi, whose 
continued importance for the comprehension of 
macro-structures and the compiling of data for the 
reform of education systems has not been question- 
ed. Mr. Klafki summed up in his report the find- 
ings of his group, whidi indicate the conlribution 
made by action researdi to defining and differen- 
tiating fundamental scientific researdi in educa- 
tion and emphasise the importance of traditional 
empirical methods for action researdi. The signifi- 
cance for action researdi of the fact that only the 
efforts to overcome a false polarisation of attitudes 
have cleared the way for further development of 
both types of researdi has become ovious from 
the example of the controversy over the isolation 
of variables. 

I now come to the third group of questions whidi 
includes the interpretation of the fundamental 
concept of "democracy", its reflection in education 
and the sdiool and participation in decision-mak- 
ing processes by teadiers pupils, parents and other 
persons closely concerned. This is what is meant 
by the term "Demokratisierung" in German; our 
neighbours content themselves with less ambitious 
^ind less overall definitious (sudi as "participation" 
in Frendi). 

The discussion on these questions has revealed per- 
haps most clearly the great differences in funda- 
mental attitudes and their relation to the historical 
traditions of the individual national states with 
their specific implications for education. 

From the relationship between researdiers and 
teachers engaged in the scientific evaluation of 
sdiool experiments it has become c^ear that the tv/o 
kinds of activity are in fact convergent and above 
all of equal value. This does not mean that the 
field should be left to. the utopiarudeas of all-round 
experts; in practice this might quidcly lead to a 
preponderance of all-round dillettrntes. 

With reference to my thesis of the dialectic in the 
three conflicts outlined here I should like to add 
a personal comment. In Eastern Europe the con- 
cept of "practitioners takii^c; pp/A in research*' has 
been introduced into discussions on educational and 
social forecasting: these practitioners include tea- 
diers. Without going into the question of compa- 
rability, I consider that this concept comprises fac- 
tors which could also become pertinent in our social 
systems. In this connection we should remember 



the need, already referred to, for teadiers, parents 
and pupils to become conversant with the funda- 
mental principles of research methodology and its 
instruments. However, the question put at the end 
of the plenary session remains unanswered: to 
what extent can the scientific nature of the work 
be safeguarded within the context of sudi a re- 
seardi-structure? 

This concludes my survey of the three groups of 
problems. Allow me to make one concluding com- 
ment whidi is connected directly with the question 
of "democratisation**. It was natural that in this 
symposium, whidi was centr^ on the open and 
controversial questions of scientific evaluation of 
pilot projects in sdiool reform, there should be 
intensive analysis and discussion of the crucial role 
of the teadier. The pupil's participation in the re- 
seardi process, an admittedly highly complex sub- 
ject of investigation and experimentation related 
to general and particular factors sudi as physical 
constitution, age group, social origin and the like, 
was necessarily left in the badc-ground if not over- 
looked. 

In making this comment at the end of my talk my 
aim is to give it its due weight. For I have learned 
from my own experience that both l3ehaviourists 
and educationalists who derive their ideas from 
radical criticism of our society make all kinds of 
ingenious statements at their meetings about sy- 
stems, methods and prospects, however much their 
individual philosophical and political standpoints 
may differ. On the other hand, the word "diild" is 
often dropped in passing, at best, if at all. I should 
like to relate this comment to two questions raised 
during the symposium. 

First there is the question of teadier participation 
in action researdi. Is it altogether impossible that 
teadiers, in generalising their own ideas, would 
loose touch with the actual learning and life situ- 
ations of their pupils and demand too much of 
them? We must see this danger and must prevent it. 

The other point is the need, mentioned by Mr. Gau- 
thier in discussion, for common cores in decentral- 
ised and individualised education. They are cer- 
tainly necessary, not only to protect the pupil 
against unreasonable strain when dianging sdiools 
but also to enable him to acquire the necessary 
general social knowledge. But over and above this 
content aspect there is the question of promoting 
learning processes which — simultaneously with 
and in addition to the acquisition of generally ne- 
cessary social knowledge and knowledge related 
to the individual — take account of affective fac- 
tors as essential to Individual self-determination. 
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REPORTS OF THE DISCUSSION GROUPS 



Group discussion I and 11 



Rapporteur: Mrs. B. GAARD£R» 
Researdi Consultant, 

The Roya! Ministry of Church and Education^ Oslo. 



Evaluation of school reform projects 

The group experienced great difficulty in respond- 
ing positively to the first theme. Following a well- 
known procedure in education, the group there* 
fore proceeded from the particular to the general 
and found wide variations in the countries repre- 
sented. Some examples follow.. 

There were projects which might, be described as 
local pilot studies of a kind which, if replicated in 
other localities and carefully evaluated, could in- 
fluence central policy regarding school organisation. 
There was an example from one country of a great 
deal of money being spent on decentralised pro- 
jects with results of so limited a nature that the 
central government was currently playing a more 
active, though not a prescriptive role. 

In other cases where national projects were tried 
out in many different localities, great attention 
was paid to the multiplicity of factors which can 
influence outcomes. 

From discussions of these and other examples the 
group reached the conclusions which follow. 

— The didiotomy between centralised and decen- 
tralised approaches was not as clear-cut as the 
title of the symposium implied. It might well be 
considered necessary in some countries to differen- 
tiate between those research and development ac- 
tivities which were appropriate to (a) centralised, 
(b) decentralised procedures. The group found it- 
self unable and unwilling to produce guidelines 
which would be universally applicable. 

— Educational policy in all countries appears to 
result from complex interactions between a num- 
ber of political, economic and social forces, with 
the findings of educational research commonly 
playing a relatively minor role. However, the group 
felt that although governments would not necessa- 
rily postpone decisions pending the results of long- 
term research, it would be valuable for them to in- 
itiate short-term feasibility studies with the aim 
of providing reliable information on Arhich to base 
future decisions. The group recoguises that, on 



major issues, educational research is but one com- 
ponent in a totality of factors influencing the de* 
cision-making process. 

— Policy decisions in the field of education are 
customarily formulated in somewhat broad terms. 
It may be decided, for example, that provision for 
nursery education should be extended to all child- 
ren from the age of three. Once a decision of this 
kind has been implemented, an important function 
of research and development work is to build up 
a many-faceted picture of the conditions and cir- 
cumstances most apt to maximise the potential 
benefits of the reform, se that any subsequent mo- 
dification of the original decision may be soundly 
based on empirical evidence. 

For example, if it had been decided that nursery 
education should be extended to all three-year-olds, 
a programme of research and development would 
be necessary in order to provide information on 
questions of detail such as: Should nursery educa- 
tion take place in separate schools or in classes/ 
units attadied to existing primary schools? Does 
nursery education exert a differential effect on 
children of differing socio-economic backgrounds? 
Can physically-handicapped children be success- 
fully integrated into nursery schools/classes? And 
so on. Well-defined, systematic and properly eva- 
luated research and development studies are essen- 
tial if broadly-conceived decisions are to be given 
necessary depth and texture. 

— An evaluator who is a member of the develop- 
ment team seems to be particularly important at 
the start of the project. His task then is to collect 
data for a quick feedback of information about the 
progress of the project. At this stage techniques of 
data-collection may lack the refinement normally 
considered necessary in rigorous research. These 
data might be qualitative or quantitative. The in- 
formation fed back to those taking part in the study 
needs to be put in an easily understandable form. 

As the project develops further external evaluation 
is also needed. The perspective in this case would 
be wider. External evaluators can underline im- 
portant factors of the innovation which have been 
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overlooked or taken for granted. External eva- 
luators might also be better able to summarize the 
results and see more possibilities for generalisation. 

— A major area of agreement within the group 
was the acceptance of the view that both in deveK 
opmental and evaluation situations the involve- 
ment by participants should be seen as being im- 
portant to the overall outcome of any specific pro- 
gramme. An analysis of the apparent failure of 
many development programmes in the past has 
resulted in increasing efforts to involve more tea- 
diers in the creative processes called for in such 
work. This experience would suggest that in cor- 
responding fashion it is important to involve as 
many teachers as possible in the evaluation pro- 
cess. Thus a major aim of the educational researdi 
programme should be to promote a greater aware- 
ness on the part of teachers, both critical and crea- 
tive, of educational issues at every level. This in 



turn should help serve to raise the whole level of 
public debate on educational matters so that im- 
portant decisions regarding the rdle of education 
in society may come to be made on the basis of 
fuller information and involving a more critical 
assessment of the issues involved. 

The group agreed that irrespective of the style of 
evaluation adopted an important function of eva* 
luation should lie in in-service training whereby 
the information gathered would be critically ex- 
amined with teadiers in relation to the programme 
under consideration. A further important rdle of 
the evaluator must be that of an impartial but 
critical commentator whose views can illumine the 
information available to the decision-makers. These 
may be teadiers involved with detailed curriculum 
matters, parents concerned with the educational 
progress of their diildren, or politicians acting in 
issues of major social importance. 



Group discussion ill 

Rapporteur: Mr. B. W. KAY. 
H.M. Staff Inspector, 

Department of Education and Science, London. 



Relationship betineen teacher and researcher 

Teadiers, once enthusiastic for reform, are now 
sometimes reacting against innovation and research, 
especially when it is imposed from without and 
when it has no obvious benefits to confer upon 
their sdiools and their pupils. Consequently it is 
sometimes difficult to obtain the co-operation of 
teadiers, and in particular of their Unions, and this 
is particularly evident when the researdi involves 
questionnaires or externally-imposed tests, which 
can be interpreted by teadiers as 'spying* on them. 
The researdier is thought of as an 'outsider' even 
if he may have only lately left the classroom him- 
self. In one country the hostility by the Ministry 
to some forms of researdi discourages teadiers from 
participating. 

The researdier can however improve the attitudes 
of teadiers in a number of ways. 

— In short-term applied researdi projects, he can 
make explicit the benefit to pupils — this is 
rightl> a top priority for teadiers. 

— He can take pains by involving them in plann- 
ing etc. that teadiers and pupils do not feel that 

. they are being used as guinea-pigs. 



— He can ensure full preparation and discussion, 
involving teadiers and others, before researdi 
is begun. Some countries have set up consulta- 
tive madiinery involving teadiens and their 
unions. 

— He can use volunteers as far as possible. 

— Some countries provide extra resources for the 
sdiools involved and extra pay for the teadiers. 
Participation in projects can also bring prestige. 

— In-service training can help to give teadiers 
confidence in tackling the unknown. 

— Researdi is most acceptable to teadiers when it 
most obviously is designed to meet the felt needs 
of sdiool or society. 



Eocperimental schools 

Some innovation can take place in some areas of 
the sdiool without greatly affecting the sdiool as a 
whole: some involves a diange in the whole struC'* 
ture. Some sdiools become 'de facto' 'experimental 
sdiools' by the enthusiasm of their staff over the 
years for taking on new projects. However the 
term is best kept for those sdiools involved in a 
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planned and properly evaluated researdi exer- 
cise. The prcblems of comparison with 'control* 
schools are severe where experimental sdioo^s at- 
tract extra resources, greater prestige and f^o bet- 
ter teadiers and moire interested parents. In some 
cases these problems are met by similarly benefit- 
ing the control schools. 



Motivation Jot research 

Teadier-felt needs are one valid basis for research, 
particularly in the field of curriculum develop- 
ment. But this can result in piece-meal work, if 
there is no long-term coordination and planning. 

Another basis is society's needs. In a time of change 
it is difficult to anticipate the needs of ten years 
hence — though this is the sort of time-scale that 
researdi must follow. But the extent and rate of 
change can both be over-stated — more remains 
stable than suffers change. 

Education serves to 

• transmit an inherited system of values — this 
remains an important and stable element; 

• teadi the skills needed for a person to live ad- 
^ equrit^ly in a sophisticated society; these include 

a range of subskills whidi may change in relative 
importance; 

• convey the non-cognitive attitudes and qualities 
needed for a mature, full and happy life in 
society; 

• prepare pupils for change by encouraging flexi- 
bility of approadi and by preparing them for 
life-iong education; 

• anticipate as far as possible future vocational 
needs — some pupils at present in sdiool will 
be in jobs which do not exist at present. In some 
countries diange is taking place too quickly for 
the results of researdi to be waited for. 

• Society in some countries is seeking reassuran- 
ce that educational institutions are achieving 
adequate standards for their pupils. Hence a 
new demand for evaluation. 



Decision making in the setting up of research 

Politicians on the one hand and teadiers and others 
concerned with education on the other are the 
agents mostly involved in interpreting the educa- 
tional needs of society, and where they conflict, in 
deciding priorities. Neither necessarily fully re- 
flects the views of a pluralist society. It is desirable 
that both should have contact with reprensatives 
of varying interests and be responsive to them. 



Apart from the basic educational requirements ac- 
knowledged by all the identification of needs is to 
some extent subjective — the interpretation of so- 
ciety's needs is basically political, from whatever 
source it comes. 

The politician is the main patron of research, but 
his motives are not always pure. Researdi can be 
called upon to support a decision already made on 
political grounds rather than to give guidance in 
coming to a decision. By financing researdi and 
awarding priorities the politician is in effect ma- 
nipulating researdi in his own interests. The re- 
seardier may moreover be called in to test an hypo- 
thesis formulated by someone else, and possibly 
expressed in a form that he would not have chosen. 
A moral dilemma can arise over the publication of 
material whose conclusions run counter to govern- 
ment policy. In some countries the researdier has 
the right to publish the jacts under any circum- 
stances, but not his interpretation of the facts. 

It is vital that autonomous sources of research like 
institutes and universities should continue to exist. 
Even this cannot however be regarded as being 
free from political bias, and in fact they may lack 
the direct responsibility that the politician owes to 
parliament and electorate. Even here the issue of 
publication may arise. 



Dissemination and its implications /or evaluation 

There appears to be an inevitable loss of efficiency 
as projects become more widely disseminated, and 
this loss of efficiency begins as soon as the project 
passes beyond a few closely involved schools. Not 
only does teadiers' motivation become less, but 
teadiers not closely involved in the setting up of 
a project often fail to understand fully the aims 
and methods involved. 

This loss can be minimised by the setting up by 
the project team of an in-service training pro- 
gramme for teadiers starting to use the material, 
by the provision of very full printed guidance for 
teachers in the form of a manual, by the encourage- 
ment of the setting up of local groups of teadiers 
working closely together. 

Motivation can also be maintained after the formal 
end of a project where teadiers — and even pupils 
— are involved in continuing assessment. This can 
also provide them with a continuous feed-bade. 

It is difficult to ensure the 'multiplication* effect 
since teachers often fail to communicate their en- 
thusiasm and expertise to others. However projects 
have to expand with limited manpower and the 
best way of maintaining the impetus appears to be 
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the forming of local groups, whidi can maintain 
closer contact between researcher and teadier. 
Guidelines from the centre may still be necessary. 



The nature of evaluation 

The range of evaluation approadies includes at the 
one end objective scientific description (whidi will 
however often imply value judgments, as the eva- 
luator brings his philosophy with him). At the 
other end it will include standardised testing of 
easily measured data with a high degree of accu- 
racy. A range o{ instruments will be needed to 
cover all the i^^oper objects of investigation, and 
different standards of accuracy v/ill be appropriate 
for different parts of the spectrum. The more com- 
prehensive the project or other object of enquiry 
the more global will have to be the approadi to 
evaluation. The evaluator will aim at the ^<t^atest 
accuracy and greatest objectivity compatibl'* wit\ 
the object of evaluation. Examples were givei* * C 
a range of different methods of evaluation being 
applied to the same situation — e.g. a series of 
accurate descriptions of observea processes could 
help to explain the reasons for different levels of 
achievement recorded in tests- 

An example was given of a feasibility study ^short- 
ly to be mounted to evaluate primary education by 
a sampling technique using three lines of assess- 
ment. 

• standardised tests; 

• structured subjective judgments by itispectors, 
using four or five point scales in a questionnaire 
prepared joirttly by inspectors and a research 
institute; 

• descriptions of educationsil processes observed 
by inspectors. 

In order to standardise the second and third as far 
as possible, "moderation" would be carried out by 
including in the team ^'isiting eadi sdiool a mem- 
ber of the team which had originally drswn up 
the criteria, or one who had worked with sudi a 
member. The purpose was to evaluate a broad 
spectrum of educational objectives. 

In action researdi there is room for both empiri- 
cal quantitative evaluation and for less precise 
measures. The researdier needs to be aware cl tne 
strength and limitations of his various tools. 

— He should make u' as far as possible of quan- 
titative tediniques, oat there are some areas of 
educational objectives whidi sudi tediniques 
cannot easily encompass. It may then be better 
to use less reliable methods rather than to ex- 

52 



pend disproportionate efforts to obtain only 
meagre additional results by quantitat:^'** me- 
thods. 

— The involvement of teadiers as collaborators 
rather than consumers brings advantages to 
co«>nterbalance seme loss of scientific precision. 
(Participant observation, incidentally, is not a 
recently introduced technique). There are areas 
in whidi the teadier actively involved can eva- 
luate objectively, but an overall judgment is 
not compatible with close involvement. There 
is a danger that an ideological bias may be im- 
porttid. 

— An insistence on precise measurement can limit 
the range of evaluation. Even instruments like 
questionnaires are fallible because of the vary- 
ing attitudes and degrees of cooperation of the 
answerer. 

— Any evaluation inflMences what is being eva- 
luated — even observation affects the situation 
observed and could falsify the data. 

— What the observer sees can be limited by his 
preconceptions. The evaluator might well adopt 
in the first place a very open attitude and only 
later attempt, to structure hio observations in the 
light of experience. 

— Observation can result in more or less quanti- 
fied information a three or five poir^t scale is a 
cmde measurement; a statement that **so mu:^y 
examples of this or that were observed" may 
be more precise. 

— Although some qualities to be evaluated are very 
difficult to define and may be impossible to 
measure, some definition, however broad, is 
needed before evaluation can be attempted. 

An example was given of an attempt to evaluate 
pupils' behaviour in a r.^mber of areas selected 
from teadiers* identification of their educational 
objectives. Qualities sudi as perseverance or cri- 
tical sense ave broken down into a series of speci- 
fic, easily identifiable items of behaviour. The in- 
tention is that a fellow-te^dier should observe a 
series of classes and note down entries on an 
observation sdiedule. It is inevitable that teadiers 
will consciously or unconsciously adapt thji'' me- 
thods to. produce examples of the behaviour under 
observation, and that the pupils too will be in- 
fluenced. This is recognised and accepted. This is 
regarded as a piece of formative evaluation which 
will tend to modify and improve teadiing tech- 
niques. 

In another country a project for the evaluation of 
five pilot sdiools *r« a number of fields of enquiry 
failed because of inadequate initial planning and 



subsequent changes of policy. In place of the glo- 
bal project the researdi team is now restricting the 
evaluation to a more limited field. In particular 
groups of volunteer teadiers, helped by psycholo- 
gists, are studying four areas of the curriculum 
and defining objectives in terms of the behavioural 
qualities whidi it is hoped pupils will acquire 
through the study of subjects, rather than in terms 
of the information content of subjects. 

In the course of discussion a range of views was 
expressed about the validity of inter-system com- 
parative evaluation. The results of sudi evaluation 
are particularly likely to be misinterpreted or mis- 
used. In practice only a few characteristics shared 
by the two systems to be compared can be selected, 
and these are frequently selected more on political 
grounds than because they offer the best basis for 
overall comparison. 

Costs of evaluation 

Evaluation involves expenditure in two ways: 

— The actual cost of the evaluation exercise must 

be proportional to its product — some expensive 



evaluation procedures have barely justified their 
cost — the same conclusions have been reached 
independently by more simple and direct methods. 
It is important that unnecessary duplication of re* 
search should be avoided — hence the importance 
of ERIC and EURISED. 

— New methods of teadiing and new forms of 
sdiool organisation arising from researdi can be 
more costly than traditional ones in building re- 
quirements, teadiing resources, in-service training 
and supply of teadiers. An element of costing for 
the implementation of a project should be built 
inio the evaluation so as to give better guidance 
for policy decisions by Ministers. 



Terminology 

It was recommended that for future conferences of 
this sort among the preliminary papers should be 
a glossary in the working languages of terms 
likely to be used. Where no verbal equivalent 
exists, a short description or definition might be 
required. 



Group discussion IV 



Rapporteur: le Recteur M. GAUTHIER, 
Inspecteur d'Acad^mie, 
Ministere de TEducation, Paris. 



The multilingual group was composed of represen- 
tatives of the following States: Austria, Belgium, 
Federal Republic of Germany, France, Italy, Nor- 
way, Spain and Switzerland. 

The problems it studied may be classified accord- 
ing to four main themes: 

— decentralised innovation, 

— action research, 

— formative and summative evaluation,, 

— evaluation techniques. 



Decentralised innovation 

Innovation may be initiated either by the central 
government or at the local or regional level. In 
the case of an initiative by the central government 
deciding to survey a reform project, either the 
same research can be entrusted to a certain num- 



ber of sdiool establishments, or different elements 
of the researdi can be distributed among them. 
Several participants stressed the importance of 
local initiative in innovation. They consider that 
the main function of the central authorities is to 
listen to teadiers, who must play an essential part 
in defining the objectives of innovation. 

The relation between the central authority and the 
units on the periphery was examined with regard 
to the nature and scope of the control exercised by 
the central authority, and with regard to the de- 
gree of freedom to innovate accorded to the local 
educational units, whidi varies according to the 
question under consideration. 

In the matter of school siructureSf reform projects 
almost always derive from the central government, 
because of the need to assure an indispensable co- 
herence in the educational organization of the 
country, and because this brings into question the 
State's general and financial policy. 
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As for teadiing content, in some countries it is im- 
posed in ^stematic fashion, while in others sdiools 
and sometimes even the classes themselves are all- 
owed to exercise choice in relation to a more or 
less important portion of the curricula. But this 
latitude is always limited by the need to prepare 
pupils to pass examinations in the subject, and by 
the concern not to place at a disadvantage the 
growing number of children who have to change 
sdiools because of family mobility. It is therefore 
necessary to impose at the very least a common 
core curriculum; on the other hand, some countries 
are turning to the concept of more flexible exami- 
nations. 

It is in the field of teadiing methods that one finds 
the greatest scope for initiative, although it should 
be noted that innovation in teaching methods can 
lead to a modification of content (for instance 
language^leaming by audio-oral method brings 
about a new type of knowledge and practice of the 
language). 

Central authority action is felt in several ways: 

— examination of local initiatives and the neces- 
sary consent for setting up the experiments, 

— possible control to prevent local initiatives con- 
trary to the country's educational policy from 
slowing down instead of promoting the evolution 
of the educational system, 

— coordination of decentralised rescardi, 

— help for local units engaged in research, 

— setting up of a reciprocal flow of information 
between the central level and the establishments 
at the periphery (exchange of information about 
experiments and results, ideas bank), 

— financing, either by the State alone, or by 
division of the expenses between the State and 
local or muncipal budgets, 

— final evaluation and generalisation decisions. 



Action research 

The group aimed to define it in contrast to the 
more classic form of development research, a pro- 
cedure consisting essentially in research limited 
to a restricted group, at the end of which the pro- 
blem of possible generalisation arises. 

The nature of action research is wider and more 
dynamic. A greater number of teams in the field 
participate in the study of an innovation project. 
These teams are made up of working teachers 
helped by specialists in the relevant disciplines 
and by research workers, who together establish 
a process of direct interadjustment of practical 
experience and theoretical work. 



Can one say that action research must be demo- 
cratic, and is this qualification compatible with 
the scientific nature of the research? I would seem 
to be a question of involving the greatest possible 
number of people affected by the reform project, 
of obtaining the full participation of teachers, 
pupils, parents and even locally elected officials, 
in fixing objectives, in elaborating methods and 
curricula, and in observing the conduct of experi- 
ments. 

Furthermore action research has a very strong 
internal dynamic, for decentralised experimenta- 
tion carried out in a large number of local units 
requires periodic comparisons by the different 
teams, who go on by means of successive syntheses 
to the progressive elaboration of objectives and 
methods. It is a question here of a continuous 
process of internal evaluation within the teams 
themselves, who are thus led to adjust their actions 
to the objectives, or to correct and modify these 
objectives in the light of difficulties encountered. 

Formative and summative evaluation 

Besides the continuous formative evaluation 
organically involved in action research, we must 
distinguish siunmative evaluation, which interests 
the central authority responsible for making 
decisions concerning the generalisation of the 
tested reform. By whom and how can it be carried 
out? Should it be carried out by external 
evaluators? The group considers that summative 
evaluation should not be divorced from continuous 
evaluation, and that if external evaluators are 
brought in, it should proceed on the basis of the 
work of teachers and researchers who have taken 
part in the experiments. It should invite their 
participation or rijk being challenged. 

It was noted that in action researdi decentralised 
cover in a large number of •*fields", this multiplicity 
of test locations is both a source of wealth owing 
to the number of contributions and a source of 
difficulties for the evaluation because of the diver- 
sity of conditions to be found in the various 
experimentation locations. Rather than provide a 
value judgment on the tested innovation as a 
whole, summative evaluation would consist in 
giving an account of the various forms of re- 
search and of the results obtained in the various 
sdiools where it was carried out, bearing in mind 
the conditions and mode of work peculiar to each. 

The group discussed the recruitment and training 
of researdi workers. Besides academics trained in 
institutes of educational science, recourse is most 
often had to psychologists, sociologists and teachers 
who devote themselves temporarily to the task. 
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since they have seldom had previous training, 
these researchers acquire this training empirically 
during the research process, which increases both 
the duration and the cost of the experiment. 

Evaluation techniques 

Following the general discussion on evaluation 
techniques, the last meeting allowed the group to 
examine concrete methods of evaluation applied in 
certain school experiments in progress. 

The impression gained from the various contribu- 
tions was that preoccupations were similar and 



that similar means had been employed. Among the 
procedures used and remarked on were the elabo- 
ration of pupil maturity tests, sociometric 
inquiries, the partenal situa*tion, and performance 
tests taken at the beginning and during the course 
of study. * 

In addition, there was agreement in acknowledging 
the need to inform everybody affected by the 
innovation as completely and as coherently as 
possible — teachers, pupils, parents — as to the 
objectives of the innovation being tried out. 



BACKGROUND DOCUMENTS 
Austria 



PART I 

THE EVALUATION OF COMPREHENSIVE 
SCHOOL EXPERIMENTS 

Within the framework of the programme for 
Austrian school development, pilot projects are 
being carried out in the schools for 10 to 14 year 
olds, according to a federal law enacted in 1971. 
The purpose of these projects is to test forms of 
comprehensive schools. 

At the centre of the experimental work stands a 
type of integrated comprehensive school which 
provides for attainment groups (setting) in mathe- 
matics, German and English from the 5 th to the 
8th grades. Heterogeneously grouped classes are 
used for instruction in all other subjects. In 
addition to this, a type of school devoted to 
•'guidance" is being tested. This is conducted like 
an integrated comprehensive school in the 5th and 
6th grades only. 

The law prescribes scientific control of the com- 
prehensive school experiments so that information 
can be gathered which is to serve as a basis for 
political decisions concerning the organisational 
set-up of the schools for 10 to 14 year olds. For the 
time being, the experimental types are to be 
tested for five subsequent years from the 5th grade 
up to the 8th grade respectively. 



by Professor G. PETRI, 

Zentrum fur Schulversuche und Schulentwicklung, 
Graz. 



Evaluation goals 

First of all, information concerning the further 
development of school types and evaluation sys- 
tems is being worked out (formative evaluation). 

In the final phase of the experiment, data are to be 
gathered on which to base a decision about the 
organisational form of the sdiools for 10 to 14 year 
olds. For this purpose, hypotheses about the differ- 
ing effects of the traditional school as opposed to 
the experimental school types are to be tested 
(<5ummative evaluation). 

For the preparation of a decision, essentially the 
following questions will have to be examined: 

— Does the new type of school organisation pro- 
vide a better education for pupils with a 
socio-economically and socio-culturally disad- 
vantaged background? 

— Do educational opportunities improve in those 
regions with an insufficient number of higher 
level schools? 

— Do pupils attain, on the average, a higher 
standard of education when strengths and 
weaknesses are taken into consideration by a 
setting systems? 

— Considering the fact that pupils of all attain- 
ment levels are together for a greater part of 
the instruction time, is it possible to develop 
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more satisfactory contacts and a better under- 
standing among pupils with varied social 
badcgrounds? 

Is the repeater problem satisfactorily solved by 
the new form of school organisation in whidi 
repetition of a class is avoided by reclassifying 
the pupil to the next lower attainment level in 
the respective subject? 

Does the new type of school organisation, which 
substitutes furtherance of the individual for the 
principle of selection, promote socially-integrat- 
ing behaviour patterns on the part of teachers 
and pupils? Does it increase the motivation of 
the pupil to learn more about the subjects? 

Does the new system, as opposed to the traditio- 
nal one, lead to a lowering of attainment on the 
part of pupils with certain levels of aptitudes? 



Methods 

The evaluation of the school experiments proceeds 
from detailed hypotheses about directly observable 
facts. Each of these hypotheses makes comparative 
statements concerning: 

— differing effects of 

— various school types on 

— pupils with specific characteristics 

— under specific educational conditions. 

The following itemisation offers a rough overview 
of the variables which come into consideration in 
the building of hypotheses: 

1. Dependent variables: 

— Career 

+ Educational level attained at the end of 

compulsory school attendance 
+ School or occupational training after the 9th 

grade 

+ Vocational or scholastic level attained later 
on 

— Cognitive variables: 

+ Attainment in the subjects with attainment 
levels (mathematics, German, English) 

+ Attainment in the subjects taught in hetero- 
geneous groups 

— Non-cognitive variables 

+ Motivation of the pupils with regard to 

learning and education 
4- Attitudes towards school 
+ Stress on the pupil, fears and problems in 

connection with school 



Effects of social learning 

• Sociometric structure of pupils* groups 

• Forms of interaction between pupils 
varying backgrounds and capabilities 

Attitudes of the parents towards school 
Teacher attitudes 



of 



2. Treatment variables 

— Traditional schools 

4- Grammar school (Gymnasium) 
4- Secondary modern school (Hauptschule), 
first stream 

4- Secondary modern school (Hauptschule), 
second stream 

— "Integrated" comprehensive school 
4- Attainment level, group I 

4- Attainment level, group II 

4- Attainment level, group III 

4- Instruction in heterogeneous groups 

— Guidance period 



— "Additive" 
school 



or "multilateral" comprehensive 



3. Control variables 

— Pupil characteristics 
4- Sex 

4- Psychological characteristics (intelligence, 

motivation, etc.) 
4- Social status 

4- Out-of-school help with studies 

4- Attainment at elementary school, degree of 
maturity for secondary school, class repeti- 
tion 

4- Milieu: town, rual area 

— Instruction factors 

4- Learning objectives 
+ Instructional methods 

• Textbook 

4- Distribution of attainment in the class 

• Mean 

• Standard deviation 

4- Group dynamics of the learning group 
4- Teacher characteristics 

• Training 

• Sex 
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• Age 

• Educational style 

• Teadiing skill 

• Attitudes 

The testing of hypotheses presupposes: 

— valid and reliable methods for measuring the 
effects; 

-- the uniform implementation of the various 
steps whose effects are to be determined; 

— a sufficiently exact control of all variables 
which influence the effects examined apart from 
the experimental variables. 

Of these prerequisites an adequate treatment of the 
control variables is the most difficult to realise in 
the study under consideration. 

The control variables either have to be kept con- 
stant, to be systematically varied, to be strictly 
randomised, or they have to be taken into account 
mathematically on the basis of measurements. 

Randomisation is of essential importance particu- 
larly for the elimination of the extremely heavy 
influence of teacher personality and the dynamics 
of the learning group. 

The "Hawthorne effect" can presumably be reduced 
by extending the school experiments over a longer 
period of time and increasing thoir number. 
Systematic interviews of teachers at experimental 
and traditional schools allow well-fo'inded assump- 
tions to be made as to the distribution of attitudes 
which might favour a "Hawthorne effect". 



Planning 

The experimental phase, in which the new organi- 
sational school types are to be tried out under 
scientific supervision, will be preceded by a deve- 
lopmental stage. 

In this developmental stage, the basic materials 
with which the teacher has to work are t-^ be 
developed and tested. These include guidelines and 
data concerning the grouping and regrouping of 
the pupils into the attainment groups, as well as 
the mode of instruction in the various attainment 
levels and in the heterogeneous class periods. 

The trials in the developmental stage are to be 
evaluated with the following goals in mind: 

— obtaining feedback as a basis fjr planning 
further developmental activities; 

— building up both an organisation and a staff ^r 
the scientific evaluation (control) of the experi- 
ment; 



— development and/or testing of methods of data 
compilation, data processing and statistical ana- 
lysis; 

determination of the conditions to be fulfilled 
to enable school experiments to be continued in 
a scientifically controlled manner. 

At the beginning of the 5th grade, the control data 
are to be gathered in an entrance survey. Later, 
many of these data would no longer be available. 
Primarily, they include psydiological characteristics 
of the pupil such as intelligence, past learning 
progress, attitudes, etc. 

Towards the end of the 6th and 8th grades data on 
the dependent variables are to be gathered (scho- 
lastic attainment, attitudes towards school, learning 
motivation, sociometric data, etc.). The treatment 
and control variables which change in the course 
of time are also to be determined, e.g. which 
attainment groups the pupil belongs to, participa- 
tion in tutorial sessions, learning help outside the 
school, etc. 

The evaluation of the school experiments requires: 

— a determination of the probability of the exi- 
stence of varied effects of the individual school 
types; 

— if possible, information on the size of differences 
in effects; 

— and, if necessary, a determination of the form 
of the functional relationships between effects, 
treatment and control variables. 



First results 

The experiments evaluated so far belong to the 
introductory phase of the overall experiment 
envisaged, both with regard to the organisational 
types tested and with regard to the methods of 
scientific control used. The first results related to 
the 5th grade of the experimental schools in the 
school year 1971/72. 

In the autumn of 1971, the Austrian school psycho- 
logists undertook a survey of 7 000 pupils in 145 
experimental school classes and in 79 traditional 
school classes. Included in this survey were intelli- 
gence and scholastic attainment tests, a question- 
naire for the pupils as well as on'i for their parents, 
and a sociometric test. In addition, the grades 
received by the pupils in primary school and the 
assessment of the pupils' maturity for the first or 
second stream of the secondary modern school 
(Hauptschule) as judged by the primary school 
teacher (who has the pupil all 4 years) were regis- 
tered. 
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An intermediate investigation was carried out in 
June 1972 and i9(3 by the sdiool psychologists. 
This was performed at the end of the 5th and 6th 
grades, respectively, on approximately 4 600 pupils 
in experimental and control schools. The pupil's 
performance in mathematics, English and German, 
as well as his attitudes, interests, and sociometric 
variables were investigated. 

First of all, the effects of school organisation on 
sdiool performance and on the equality of educatio- 
nal opportunities were analysed ('). 

The following hypothesis was tested: 
Sdio^astic attainment of the pupils in the 5th grade 
in the ability-differentiated subjects (mathematics, 
English, and German) is, on the average, greater in 
the integrated comprehensive school than the 
attainment of comparable pupils in the traditional 
sdiool system, 

since: 

pupils with equal aptitudes learn approximately 
the same amount in the appropriate attainment 
groups of the comprehensive school as m the 
traditional school (sub-hypothesis 1); 

however, pupils in the comprehensive sdiool are 
better allocated to attainment groups coinciding 
with their respective aptitudes than in the traditio- 
nal school system (sub-hypothesis 2); 

equally talented pupils learn more in homogeneous 
learning groups corresponding to their respective 
levels of attainment than in classes less well suited 
to their respective levels of attainment (sub- 
hypothesis 3). 

Sub-hypothesis 1 was confirmed by the experimen- 
tal results gathered to date: the upper level groups 
of the comprehensive school attained equally high 
results in mathematics, English and German as 
would be expected from equally talented classes of 
the Gymnasium. In evaluating these results, one 
has to consider the fact that various factors which 
could influence pupil performance in favour of the 
experimental school or in favour of the Gymnasium 
were not controlled in the present study; or, if so, 
only insufficiently. Such factors include: promotio- 
nal measures to assist the pupil either at school or 
at home, the family, learning motivation, sequence 
in whidi the subject matter was taught, method of 
instruction, effectiveness of the teacher, number of 
pupils in the various classes, Hawthorne effect, etc. 

It is not possible, at the present time, to form a 
scientifically sufficiently well-founded judgment 
on whether the uncontrolled factors, as a whole, 



(1) A report on the non-cognitive effects, is at present 
being prepared. 



biased the results in favour of the experimental 
schools, or in favour of the traditional schools. 

Sub-hypothe( was also confirmed. In fact, it 
was determine, chat there are often considerable 
intra-individual ability differences in the three 
subjects examined. Approximately half of the 
pupils could not be put into the same groups in 
mathematics, English and German, with three 
attainment groups for eadi subject. 

It was observed that there is considerable changing 
up and down between the attainment groups in the 
5th grade. Approximately 26 per cent of the pupils 
changed sets in mathematics, English, or German 
from beginning to end of the school year. 

The reclassification carried out in the 5th grade, 
however, did not seem to have essentially improved 
the allocation of the pupils to attainment groups. 
Thone pupils transferred to higher groups showed 
no more marked improvement in performance at 
the end of the school year than did those pupils 
transferred from higher attainment groups to the 
lower levels. ' 

This is considered as a demonstration of a need to 
base future reclassification procedures on psycho- 
logical tests. These are already being developed. 
The new technique is being evaluated at present. 

A clear statement as to whether a more exact 
placement of the pupils in attainment groups will 
improve their scholastic performance (sub-hypothe- 
sis 3), cannot be made on the basis of the data of 
the study. It is possible that the influence exerted 
by the degree of homogeneity of the learning group 
on individual performance will become more appa- 
rent in later grades. / 



Further evaluation plans 

The major problem which needs to be solved is 
whether and how the factors so far not yet suffi- 
ciently controlled can be effectively controlled in 
the future. 

The following points, to mention only a few, need 
to be considered: 

— If one wanted to eliminate the uncontrolled 
influence of the school factors which do not 
depend on the form of organisation, one would 
have to set up parallel classes in both the 
traditional and experimental schools which were 
the same in teaching objectives and methods, 
materials used, and in assessment procedures, in 
so far as differences in these respects are not an 
unavoidable consequence of the compared school 
models. 
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^ If one were to dispense with parallelism in the 
experimental and control sdiools beyond the 
selection and organisation of subject matter to 
be taught, then this would make it easier to 
make rough .comparisons of attainment. In this 
case» the effects of sdiool organisation would 
not be determined separately from the effects 
of curricular factors. One would need to 
examine exhaustively whether data gathered 
in this manner would be usable as a basis for 
decisions. 

— It is practically impossible to control the teacher 
factor by strict randomisation. The secondary 
modern sdiool teadiers (Hauptsdiule) taking 
part in the experiment, however, can be con- 
sidered as at least an approximate random 
sample if (a) they have no influence on the 
selection of their sdiools as experimental or 
control sdiools; and (b) they are not teadiing in 
the experimental classes on a volunteer basis. 

It is almost unavoidable that the group of 
Gymnasium teadiers instructing in the top 
attainment groups in the experimental sdiools 
deviate considerably from the essential diarac- 
teristics of a random sample. Their co-operation 
is entirely on a voluntary basis. 

— If it is to be determined how pupils who would 
normally attend the Gymnasium in the traditio- 
nal sdiool system would learn and behave in a 
comprehensive sdiool type, "experimental is- 
lands'' would have to be established in whidi 
the pupil would attend an experimental sdiool 
instead of a Hauptsdiule or Gymnasium. 

— Finally, one would have to ensure that .all 
points of view relevant to decision-making 
would be taken into consideration in the evalua- 
tion of the sdiool experiments. For this purpose, 
the responsible groups would have to clarify the 
question as to whether information about the 
pupils' learning progress and well-being and 
about the contribution of the compared sdiool 
models to the realisation of equality of educa- 
tional opportunity, is sufficient as a basis for 
decision — or whether additional information 
would be necessary. 



PART II 

LEARNING TESTS 

Every comprehensive discussion on the question of 
measurement of progress should proceed from the 
fact that sdiolastic attainment constitutes only one 
of several educational objectives. 

Within the framework of the development of 
measurements of progress, the following objectives 
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should be considered, in addition to sdiolastic 
attainment: 

— tediniques of learning and of problem-solving, 

— interests and attitudes, 

— self-assessment and self-esteem, 

— independence in thinking, evaluating and 
decision-making, 

— ability to experience (e.g. not only formal, 
historical, or sociological studies of literature, 
music and the fine arts, but also the ability to 
enjoy pleasurable experiences), 

— ability to co-operate, to cope with conflicts, and 
to establish satisfactory personal relationships. 

A scientifically-based development in the area of 
assessment presupposes both sufficient insight into 
the mutual relationships between assessment and 
educational objectives, and a determination of the 
various functions of assessment in school and 
society. 

The measurement of sdiolastic progress essentially 
has to fulfil the following three kinds of functions: 

— providing data as a basis for decisions about the 
juture educational or vocational dioices of the 
pupil (decision-making function); 

— fostering learning (didactic function); 

— furnishing basic materials for effectiveness 
tests and analyses of sdiolastic measures and 
facilities (evaluative function). 

The traditional system of measuring progress 
(assigning grades) fails especially with regard to 
its didactic functions. It often has a detrimental 
influence on the learning process, and seems to 
influence the development of the pupil's personality 
negatively. 

Sdiolastic attainment testing should serve directly 
to promote the learning processes of the individual 
pupils; and, at the same time, deliver information 
to the teadier for more effective instruction. 

The functions whidi concern the pupil directly are: 

— the awakening or the strengthening of the 
desire to attain sdiolastic objectives (motivating 
function); 

— the independent, effective planning of learning 
activities (controlling functions); 

— the starting of learning processes (actualising 
function); 

— the development of personality diaracteristics, 
as e.g. ability to learn, confidence in success, 
independence, interest in the subject matter 
(educative functions). 
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An effective system of scholastic attainment testing 
requires special methods and test materials. These 
have to be integrated into the instruction in such a 
way that the motivating, controlling, actua Using 
and educative functions can become fully effective. 



Motivation 

Attainment testing for instance, could fulfil its 
motivating functions more effectively by means of 
the following procedures: 

1. Immediately after the introduction of a new 
topic — e.g. by a lecture on the part of the 
teacher, a class discussion, an experiment, group 
or individual work — the pupils are given the 
opportunity to apply what they have just learn- 
ed to the solution of test problems. Subsequent- 
ly> they evaluate, their work themselves and 
determine their progress in every detail. 

It may be assumed that 

— many pupils, in such a situation, will pay 
particular attention to introductory instruc- 
tion since they know that they will imme- 
diately be able to apply what they have just 
learned; 

— many pupils will want to remove gaps in 
their knowledge or understanding as soon as 
they become aware of them; 

— every success which becomes immediately 
apparent to the pupil represents an incen- 
tive for further learning; 

— in the course of time, some pupils will deve- 
lop playful or "sportive" attitudes which 
promote a healthy motivation to learn; 

— some pupils will readily take advantage of 
the opportunity to practise when they know 
they will later need to take tests in which 
grades will be assigned. 

2. After the tests have been corrected, the teacher 
asks for a show of hands to determine which 
problems were particularly difficult, and then 
uses learning activities adjusted to the respec- 
tive difficulties. Subsequently, the pupils receive 
additional test items as homework which are 
relevant to the topic being studied. These items 
are designed to offer the pupil further oppor- 
tunities to practise. Since he can immediately 
evaluate his work, he can determine whether 
or not he has readied the formerly unattained 
objectiv*»s. 

In a subsequent period of instruction a third 
and final test is given on the subject matter, of 



which the pupils have by then a better grasp. 
This test, too, is immediately corrected by the 
pupils. 

— It may be assumed that the pupils will apply 
themselves with considerable interest to 
learning activities immediately after having 
recognised the difficulties in the first test. 
This is especially true since they know they 
will be able to close the gaps in their know- 
ledge or understanding and that they will 
be able to apply what they have just 
learned as soon as they tackle their home- 
work. 

— The pupils are motivated to do their subse- 
quent homework conscientiously if they 
really want to achieve the goals they have 
not yet reached; and if they want to see how 
much more they have learned since the first 
test. 

— Following the evaluation of their homework, 
the pupils attempt to close the remaining 
gaps in their knowledge and/or understand- 
ing by thinking over or practising the tasks 
not yet mastered, since they will want to 
achieve good results in the third test 
scheduled for the next class period. 

3. The pupils enter all test results onto a "pupil's 
record sheet" which, by means of a graph, 
shows their scholastic progress both from test 
to test within the learning unit, and from 
learning unit to learning unit. The pupil's 
record sheets also provide continuous informa- 
tion for the teacher about the individual pupil's 
learning progress. They are rot used as a basis 
for grading. 

The scholastic progress which can regularly be 
read of the pupil's record sheet can serve as an 
effective reward to the pupil for his efforts and 
should stimulate him to build long-term goals. 

4. The test problems are adjusted to the pupiFs 
abilities. In general, none will be able to 
complete all test items correctly the first time 
through the test, but all pupils are able to solve 
a considerable number of the items. Thus, the 
experiences of failure and discouragement are 
largely avoided. Each pupil can feel confident 
about his future success and satisfaction with 
his learning progress, even though he usually 
still needs to learn several things in the learning 
unit after his first time through the test. 

The shortcomings of the traditional form of 
instruction stand out clearly when we consider 
the possibilities for formative attainment test- 
ing described above. 
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— The pupil is often not motivated to follow 
the class instruction if he does not expect 
an examination or a test until much later, 
and if he does not know whether it will 
deal with the particular topic being covered 
at the moment. 

— Even classes with active participation on 
the part of the pupils (discussion, question 
and answer games) will probably not produ- 
ce the same motivation as scholastic attain- 
ment testing, since these forms of instruc- 
tion appeal to the individual pupil only 
sporadically and superficially. 

— In the traditional type of instruction, the 
pupil is confronted by a vaguely recognized, 
threatening pile of learning tasks which he 
would be most inclined to "forget". By a 
well-organized system of scholastic assess- 
ment he would be confronted by a well- 
defined set of goals. He would also be able 
to determine to what extent he had already 
achieved them and how he could learn to 
master them better. 

— In the traditional form of instruction, the 
pupil is not informed about his individual 
learning deficiencies promptly and systema- 
tically. As a result, he cannot develop the 
desire to overcome these deficiencies, and 
he runs the risk of gradually losing his foot- 
hold. 

— In most cases, the results of the pupil's in- 
dividual activities are not directly apparent 
to him. Immediate rewards are therefore 
not available to motivate his learning. 

— School grades contribute very little towards 
th^ development of intrinsic motivation to 
learn more about a subject, since they re- 
present essentially only a global index of 
success. Instead, grades develop general at- 
titudes towards attainment, success and 
prestige of doubtful mental, hygienic and 
educational value. 

— Low-adiievers almost always receive only 
poor grades. They are predominantly nega- 
tively motivated towards school and learn 
even less, since they experience more failur- 
es than successes. In this way, they get into 
a vicious circle from which it is hard for 
the pupil to break out. 

— The unfavourable learning climate which is 
so frequently observed in the second stream 
of the secondary modern schools (Haupt- 
schule) is probably caused by this very 
medianism. This could be eliminated by 
regular feedback to the pupils of their suc- 
cesses. 



Control of learning activities 

A system of scholistic attainment testing can con- 
tribute to the effective control of learning pro- 
cesses by continuously setting concrete goals for 
the pupil corresponding to his particular stage of 
learning. 

— The tests are prepared on the basis of a well- 
grounded selection and structuring of the learn- 
ing objectives. This means that the learning 
activities may be directed towards clear, rele- 
vant, and effectively sequenced objectives. 

— The learning objectives are adjusted to the 
respective ability levels of the pupil in such a 
way that he is able to master them. 

— The fact that success and failure are promptly 
fed back directs the learning activities of the 
pupil to the objectives not yet mastered. 

In the traditional form of instruction, the pupil 
receives feedback about his successes in a more 
global way. The teacher either says that he has 
learned diligently, or that, with regard to a field 
of learning whidi is not very clearly defined, he 
has to learn "more". Under these conditions, it is 
difficult for the pupil who is behind in his learn- 
ing to purposefully organise the necessary learn- 
ing activities without outcide coaching. 

Actualisation of learning processes 

The usefulness and the extent of application of 
formative attainment testh;g may be considerably 
larger if the tests are constructed in such a way 
as to serve not only as measuring instruments but 
also as learning material. In sudi a case, the measu- 
rement does not take up additional time, since the 
test serves both as a measuring and as an instruct- 
ing device simultaneously. 

If the formative tests did not also fulfil instruc- 
tional functions, then the teacher would be con- 
fronted with the difficult task of having to consider 
large amounts of test data in planning classroom 
instruction. By the integration of the measuring 
and the instructing functions in "learning tests", 
the teacher can largely be relieved of routine teach- 
ing activities. Thus he gains elbow room for treat- 
ing learning difficulties of individual pupils and 
for planning the subsequ«^nt instruction. 

Practical experience already available shows that 
it should be possible to develop the use of the 
tests as an especially effective learning process. In 
this way, formative attainment testing can be 
smoothly built right into the instructional process. 

The taking of a test will actualise learning pro- 
cesses most effectively if the series of problems 
contained in the test is organised around a hier- 
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archy of learning objectives. These, in turn, are 
to be adjusted to the existing knowledge and abi- 
lities of the pupil, so that he is able to "climb the 
ladder** of learning objectives. 

These learning processes take place with the in-* 
tensive participation of the pupil, who has to solve 
the problems not only by reproduction, direct ap* 
plication or trivial transfer of the material learn- 
ed, but also through guided discovery. If the pupil 
meets up with exceptional difficulties, he receives 
help whidi enables him to continue working in- 
dependently. 

In contrast to the traditional instructional pro- 
gramme, the pupil is not confined to small steps 
and narrow learning paths. In addition to this, 
there is often feedback only after the completion 
of a whole train of thought rather than after eacn 
small learning step. Learning tests are integrated 
into the rest of the instruction and can be more 
readily modified by the teacher, both with regard 
to the mode of application and internal organisa- 
tion. 

The possible applications of learning tests are 
manifold and range from the automation of know- 
ledge and skills to concept and principle learning, 
and to the training of problem-solving. The educa- 
tional significance of the learning tests varies from 
subject to subject. 

In traditional instruction, it often happens that 
the pupil — possibly from lade of interest, distrac- 
tion, satiation, or difficulties of understanding — 
"turns off" from time to time. Or the pupil follows 
instruction only passively, i.e. he listens and re- 
tains some things, but fails to establish a connec- 
tion between what he has heard and what he al- 
ready knows about the same subject. This passive 
receiving leads to a momentary collection of un- 
related fragments of knowledge rather than to 
the development of the cognitive structure. Only 
learning results integrated into the cognitive struc- 
ture are usable and will be retained. 

Learning tests may well be a suitable means of 
remedying these deficiences in the traditional form 
of instruction. They promote a concentrated, assi- 
milating type of learning. Thereby, these tests can 
make an effective contribution to developing the 
cognitive structure if they are will selected and 
the learning objectives are appropriately structured. 
With regard to the mode and extent of use of learn- 
ing tests, the question arises as to what weight is 
to be given to an efficient pre-programming of in- 
struction on the one hand and to the provision of 
sufficient opportunity for autonomous learning on 
the other hand. The relative educational impor- 
tance of these two principles and the possible ways 
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of their integration into the framework of an edu- 
cational system using learning tests will have to 
be investigated. One would have to proceed from 
the assumption that neither efficient pre-programm- 
ing nor autonomous learning should have absolute 
priority. Like all things in education, these prin- 
ciples have to be evaluated from case to case to 
determine the extent to which they contribute to 
the realisation of the different learning objectives. 

These ultimately have to be based on scientifically 
founded political decisions. 

Development of peTSonality characteristics 

Learning tests can presumably: 

— contribute towards developing the capacity for 
"oriented learning"' and 

— towards awakening interests in specific subject 
matter; 

— encourage the pupil to think independently; 

— strengthen the pupil's self-confidence. 

Oriented learning is characterised by knowledge 
extending beyond the momentary learning task, 
to a comprehensive learning situation. 

It involves, among other things, orientation to- 
wards: 

— the degree of actual and desirable mastery of 
learning objectives; 

— the learning objectives whidi are hierarchically 
subordinate, co-ordinate or superordinate to the 
learning goals of the moment; 

— as well as the various stages, kinds and perio- 
dic sequences of learning and thair significance. 

Several necessary prerequisities for the develop- 
ment of oriented learning can be created with the 
help of learning test systems. They include: a bet- 
ter overview of what the pupil has already learned* 
gathering experience in setting learning objectives, 
effects of learning methods, organisation of learn- 
ing activities, etc. 

Instructional techniques whidi combine the view- 
points of formative attainment testing with those of 
autonomous learning seem most suitable for the 
full development of oriented learning. The point is 
that the pupils must use their own initiative and 
creativity as mudi as possible in working with the 
learning aids. 

A good orientation towards learning and even a 
play-like occupation with the subject matter — as 
can be realised by means of formative attainment 
testing — involving frequent successful experience, 
offers a favourable prerequisite for the development 
of self-confidence, interest in the subject matter 
and independent thinking. 
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Federal Republic of Germany 



Selected projects of evaluation of school reform 



INTRODUCTION 

Evaluation of sdiool reform pilot projects in the 
Federal Republic of Germany is organised in va- 
rious ways, in keeping with the federalistic struc<* 
ture of the entire educational system in the eleven 
Lander and guided by the educational and political 
principles held by the individual state governments. 
Instead of an analysis of the complex overall situa- 



tion in the field of evaluation of sdiool reform pilot 
projects, the following summaries of the work of 
institutions practicing evaluation are presented 
here. They offer a representative view of the over- 
all situation in the Federal Republic of Germanyi 
differing as they do with respect to their official 
status and the organisational structures of the in- 
stitutions in question, as well as with regard to 
the type and extent of reform projects described. 



BAVARIA 

Evaluation of school reform pilot programmes at comprehensive and all-day schools 

by D. KRYWALSKI, 
Bayerisches Staatsministerium 
far Unterridit und Kultur, Munidi. 



Evaluatk>n of sdiool reform pilot projects at com- 
prehensive and all-day sdiools has been conducted 
centrally in Bavaria since September 1970, by the 
"Project Group on Evaluation of Sdiool Pilot Re- 
forms'* at the State Institute of Educational Re- 
seardi and Planning at the order of ^he Bavarian 
state Ministry for Education and Culture. Its job 
is to work out a basis on whidi to make decisions 
on the future organisation of sdiools in Bavaria by 
1976. This means that pilct schools must be given as 
mudi freedom as possib2^* in conceiving their re- 
forms, freedom whidi must be limited nc:ther by 
administration nor by "consulting" scientists, and 
that evaluation must be regarded as an efficiency 
dieck on specific measures taken at comprehensive 
and all-day sdiools. 

The project group does not advise on the pilot re- 
forms, because the project group, although headed 
by an educationalist working with four sociologists 
and two psydiologists, does not have the expertise 
necessary to give sd\ools advice on practical work. 
For future experimental work, however, there are 
plans to found a consultant group at the State In- 
stitute of Sdiool Educational Theory. 

In the F^-eestate of Bavaria, tht; .following pilot re- 
forms are being conducted at comprehensive and 
a!!-day sdiools: 

3 experiments at integrated comprehensive sdiools 
(North Munidi, Treuditlingen, Hollfeld). 



1 experiment at a partially integrated comprehen- 
sive sdiool (Sdiwabmuendien). 

8 experiments at multilateral comprehensive sdiools 
(Ebern, Feuditwangen, Fuerth, Geretsried, Grafen- 
au, Marktoberdorf, Nuremberg, Sdiongau). 

16 experiments at all-day sdiools (Christophorus 
Sdiool in Berditesgaden, Erlangen-Spardorf Gram- 
mar Sdiool, Gardiing Grammar Sdiool, Gefrees 
Intermediate Sdiool, Hilpoltstein Grammar Sdiool 
and Intermediate Sdiool, Primary Sdiool on Hodi*- 
strasse in Munidi, Primary Sdiool on Thelottstrasse 
in Munidi, North Munidi Comprehensive Sdiool, 
Neusaess Grammar Sdiool, Niederalteidi Grammar 
Sdiool, Nymphenburg Grammar Sdiool, PuUadi 
Catholic Family Institute, Sdiwabmuendien Com- 
prehensive Sdiool, Treuditlingen Comprehensive 
Sdiool, Untergriesbadi Grammar Sdiool). 

In order to estimate measures taken at compre- 
hensive sdiools, in particular, control schools were 
selected from the tripartite sdiool system where 
studies were conducted that permit comparisons 
between the comprehensive sdiool and the control 
sdiool. From the tripartite sdiool system, grammar 
schools, intermediate sdiools and secondary modern 
sdiools in Noerdlingen, Krumbadi, Koetzing and 
Gem(inden, as well as individual classes at Munidi 
grammar sdiools, intermediate sdiools and secon- 
dary modem sdiools were used as control groups. 

63 



6ij 



The varied and balanced spectrum of different 
experiments serves to lead a very broad basis of 
experience to the restilts, 'io that the consequences 
for educational policy — whidi must be drawn in 
any case — are not based on coincidences, but 
rather on sound practical experience. Here, eva- 
luation in the sense of the recommendations for 
comprehensive sdiools of the Educati' nal Com- 
mission of the German Educational Council differ 
basically from those for all-day sdiools. Whereas 
the scope for evaluation of comprehensive schools 
concentrates primarily on determining the effects 
of measures taken specifically at comprehensive 
sdiools and contrasts these effects with comparable 
phenomena in the tripartite sdiool system, the 
evaluation of all-day sdiools is concerned with 
case histories, discovering the attitudes of parents, 
students and teachers toward comprehensive sdiool 
measures and with economic aspects. 



The. question complex for evaluations of com- 
prehensive schools 

The overall task of the project group on evaluation 
of sdiool pilot reforms consists of control by scien- 
tific means of the success or failure of comprehen- 
sive sdiool measures carried out at multilateral and 
integrated comprehensive schools in Bavaria. Here 
the objectives of comprehensive schools represent 
at the same time the criteria for an evaluation of 
measures. The evaluation is concerned mainly with 
three objectives: 

Objective 1: 

More ability-oriented school career decisions 

A study is being conducted to determine whether 
any dianges in sdiool career decisions made by the 
parents of the pupils in the feeding areas of Bava- 
rian comprehensive sdiools can be traced back to 
establishment of comprehensive sdiools. Particular 
attention is paid to the expectations of the parents 
with respect to comprehensive schools, previous 
achievement abilities and social background of the 
pupils. 

If sdiool career decisions made by the parents for 
the pupil are altered, then changes should become 
obvious when the pupil transfers to the fifth 
grade. Of course, changes also occur at other points 
of distribution, e.g., when transferring to the 
seventh grade, but these are characterised more by 
the achievements of the pupil than by the will of 
the parents, i.e., they are influenced more by the 
sdiool system — perhaps assignment to certain 
courses on the basis of achievement — than by the 
parents. 

64 



The most radical dianges in sdiool career decisions 
by parents are to be expected in the feeding 
regions of comprehensive sdiools with an integrated 
guidance period, because no decision in favour of a 
particular type of sdiool must be made cn entry 
into the fifth grade. In the feeding areas for these 
sdiools, therefore, the desires of the parents with 
regard to sdiool completion and their transfer 
decisions at the beginning of the fifth grade after 
establishment of the comprehensive sdiool are 
compared with transfers made before the sdiool 
was established. 

In the case of multilateral comprehensive sdiools, 
parents must decide in favour either of grammar 
sdiool or secondary modern sdiool when the child 
enters the fifth grade. For this reason, no statis- 
tically provable dianges in parents* sdiool career 
decisions are to be expected. 



Objective 2: 

Progress in individual adiievement 

Learning achievement is measured with the aid of 
an objective test for all types of sdiool made up by 
the German Institute of International Educational 
Research given during the guidance period in order 
to make a comparison of the initial positions and 
rates of learning increase in the individual achieve* 
ment fields (vocabulary, reading comprehension, 
linguistic analogies, reading for information, ma- 
thematical relationships) 

— between the various sdiool systems 

— between achievement courses of the integrated 
comprehensive sdio'^ls and grammar school or 
intermediate sdiool classes. 

The inherent intellectual abilities of the pupils are 
measured with the aid of the PSB intelligence test 
in grades 5, 7, and 8. Initial positions are described, 
changes in adiievements on intelligence tests from 
the fifth to the seventh and from the seventh to 
the eighth grades determined and the following 
comparisons made: 

— the various school systems with one another 

— achievements of the integrated comprehensive 
school with the grammar or intermediate school 
classes. 

As there are no achievement tests for the seventh 
to tenth grades which may be applied to all types 
of school, the Bavarian State Ministry of Educa- 
tion and Culture has formed a working group under 
the direction of the head of the "Project Group on 
Evaluation of School Pilot Reforms" which is to 
determine and descri!)e pupil achievements at 



integrated comprehensive sdiools in German, Eng* 
lish and mathematics (on whidi performance varies 
widely) and to test the method of evaluation. 

Objective 3: 
Social integration 

For "social integration**, the primary question is 
whether the comprehensive sdiool eliminates the 
divisive effect of factors sudi as achievement, 
- achievement differentiation, type of sdiool and 
social class on informal relationships between the 
pupils. 

Supplementary data checks 

The project group on evaluation of sdiool pilot 
reforms conducts studies, in addition to the three 
criteria-oriented studies in four fields characterised 
by the fact that they cannot be assigned to one 
single comprehensive sdiool objective, but belong 
descriptively, as well as analytically, to all objec- 
tives. 

— The social badcground of the pupils and their 
abilities are registered separately from all 
questions relevant to the investigation at all 
integrated, partially integrated and multilateral 
comprehensive schools and control schools. The 
various intelligence data and social-class index 
values are obtained for all sdiools, grades and 
setted courses, and distribution patterns are 
made up. 

— By systematically studying the organisational 
structures of the pilot reform schools and by 
descriD4ng measures taken, descriptive data are 
collected on the distribution of pupils to secon- 
dary modern, intermediate and grammar schools 
in the seventh grade resulting from differences 
in the form tak^n by the guidance period, and 
a dieck is conducted to find out whether the 
parents accept the recommendations made ^t 

. the end of the guidance period. The subsequer t 
sdiool careers of the pupils are followed in orde • 
to permit conclusions as to the reliability '>f 
the recommendations. In this connection, dian- 
ges of level at multilateral comprehensive 
sdiools and dianges between grades in setted 
subjects at integrated sdiools are recorded, the 



social structure of the courses is described and 
the percentages of pupils remaining with one 
course and switdiing courses sre computed. 

— In addition to objectives and organisation, sub- 
jective evaluations of organisational conditions 
are studied, i. e., descriptions are given of how 
pupils, parents and teadiers evaluate the com- 
prdiensive sdiool measures. To this end a 
teadier and parent poll was taken* The pupil 
poll was taken during the sdiool year 1973/74 
in the fifth, seventh and ninth grades and is 
meant to give information about how the pupils 
evaluate 

• the frequent diange of classrooms (for class- 
room diange in stable/flexible differentia- 
tion) 

• the frequent diange in group composition 
(for stable/flexible differentiation) unde!:- the 
aspect of formation of friendships 

• the diange of teadiers when adiievement 
course is dianged (for stable differentiation). 

— As part of the project "Development of a Model 
for Co-operation between Researdi and Sdiool*', 
experience in co-operation between the experi- 
mental sdiools and those who carry out the 
scientific evaluation in Bavaria is recorded and 
analysed with an eye to possible improvements 
in the flow of information and co-operation. 

Studies on evaluation at alUday schools 

Evaluation of sdiool pilot reform projects at all- 
day sdiools began with a descriptive inventory and 
is being continued with empirical studies. Here the 
parents of all-day pupils are polled on their eva- 
luations of the sdiolastic situation of their diildren 
and their ideas about and expectations from the 
organisation of the all-day sdiool. Parents were 
also requested to state their attitudes on leisuretime 
activities and homework to be done at sdiool. A 
separate project supported by the Federal Minister 
of Education and Science analyses "Economic As- 
pects of the all-day Sdiool" and is to give informa- 
tion on the effect of the length of the sdiool day 
on personnel and material expenses and attempts 
to work out "economically optium approximations 
for all' day sdiools". 
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NORTH--RHINE WESTPHALIA 

Evaluation research at the North-Rhine Westphalia Sixth Form College at the 
University of Bielefeld 

by Mrs. E. PAREY, 

Padagogische Hochsdiule, Hannover. 



At the North-Rhine Westphalia Sixth Form Ccliege 
at the University of Bielefeld (OS), researdi, teadi- 
ing and curriculum development in the sense des- 
cribed below are so inseparably connected that the 
term **Begleitforsdiung*' (evaluation research) is 
insufficient to describe the researdi aspect of OS 
work, ueseardi is not an accompaniment to the 
project; rather it is an integral part of it. Thus, all 
scientific collaborators are involved in both teadi- 
ing and researdi, although the amount of time de- 
voted to eadi varies. There is no researdi plan for 
the OS; instead there is a plan listing development 
projects from whidi researdi problems arise. 

At the OS, thorough-going action researdi is con- 
ducted. This means that the traditional division 
of roles into researdi objects and research subjects 
is gradually being eliminated. 

In order to attain this objective, training pro- 
grammes are conducted whidi, in the context of 
concrete lesson plans, teadi abilities necessary for 
their empirical testing. e.g., designing of teadiing 
instruments and classroom observation. College 
students, teadiers and co-workers in special researdi 
fields will in turn be the objects of studies leading 
to the solution of problems arising from practical 
teadiing at the OS. The results of these studies are 
directly transformed into educational action. 

To be able to give an outline of the planned re- 
seardi activities at the OB, I must first describe 
the scope and objectives of the OS. The OS is 
planned as ^< link between the top grammar school 
form and Introductory university level. Its pur- 
pose is to work out conditions and solutions per- 
mitting a co-ordinated, relatively flowing, conti- 
nuous transition from the general courses of Upper 
Secondary Sdiool to specialised basic university 
courses. In a period of four years, college students 
are led from the beginning classes of Upper Secon- 
dary Sdiool (grade eleven) to the intermediate exa- 
mination in one or two university disciplines. 

The OS is conceived as curriculum workshop. Its 
aim is to develop and test curriculum units and 
sequences, new methods of teadiing and learning, 
counselling models, evaluation procedures, ^^iWe 
participation etc. in sudi a way that they l a be 



transferred as elements to institutions with com- 
parable pupil and student populations. 

The OS is concerned with introduction to sciences. 
Its aim is to develop and test university-related 
training courses for a broad spectrum of scientific 
disciplines. Introduction to science and the general 
learning objectives connected with this notion are 
thus the foc\!s of ali teadiing efforts. 

Realisation of these basic OS intentions is to be 
made possible by various forms of course organisa- 
tion: 

A well-grounded dioice of the main subject and the 
vocational field is to be prepared by participation 
in elective courses (6 hrs/week per semester) in 
two scientific disciplines from the flrst year at 
college onwards. Opportunities and problems con- 
nected with interdisciplinary projects are to be 
discovered through work in a general course (three 
weeks per semester). All types of courses serve the 
objective of a general introductory training course, 
but the supplementary course (5 hrs per week twice 
a semester) does so in a special way. Here the in- 
dividual student is mainly concerned with disco- 
vering the systematic context in whidi his elective 
subjects belong. Via analysis and teadiing of the 
basic structures of scientific processes and their 
functions in society, students learn what science 
can do to solve whidi problems and with which 
means. Eadi semester, the student has a certain 
period of time in whidi to organise his own Indi- 
vidual curriculum, the so-called intensive phase 
(five weeks per semester). He may pursue special 
individual interests and requirements and fill 
knowledge gaps. Two intensive phases must be 
used to promote his language abilities through In- 
tensive language courses. Required practical cour- 
ses (three four-week courses during the first three 
years) are supposed to emphasise and insure the 
practical orientation of tralning^ received at the 
OS. Students who have not concluded an appren- 
ticeship or have not pursued an occupation for any 
length of time must attend these practical cour- 
ses. They are prepared and evaluated in classroom 
situations in whidi students with occupational ex^ 
perience also participate. 
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The OS will open in September 1974. 220 students 
will study at the OS during the fiir^t Sw-mester. 
Thirty-five teadiers will conduct courses, plan 
lessons and conduct researdi in the context of their 
courses during the first year. Two co-workers in 
special fields of classroom research will initially 
work almost exclusively on research projects and 
conduct training programnres in researdi tedini- 
ques. Later, however, these co-workers will take 
over more teaching duties and use only part of 
their working time for researdi, when a larger 
proportion of teachers and students invest more 
time in researdi. The OS is to be regarded as a 
field for further training of teadiers and resear- 
chers in whidi Ihey learn to recognise which que- 
stion compk'xes are important in the college con- 
text and require specific study. 

If we consider the scope of the OS and the plans 
for its realisation in the light of research method, 
we discover that the emphasis of researdi activities 
at the OS is placed on formative and summative 
evaluation. The formative evaluation of the pro- 
ject will naturally be in the foreground for the 
first few years. Formulations for learning objec- 
tives will be analysed and defined* Lesson planning 
and methods for testing achievement wul be relat- 
ed to teaching objectives and evaluated as to their 
suitability. Conversion of plans to courses and the 
effect thereof on students and teadiers will be 
observed (self-observation and observation of 
others) and determined with structured and non- 
structured instruments. The results of the obser- 
vations and measurements will be processed and 
rapidly signalled back so the lesson plans can be 
modified if necessary. Simultaneously, methods and 
instruments for evaluation will be tested for sui- 
tability and further development. 

Supplementary courses in particular will be the 
subject of detailed analyses during the first year. 



This type of course makes considerable demands 
on both teachers and students, because there exist 
neither tested contents nor teaching concepts, which 
clearly shows its perimental character. Conduct ^^nd 
results of instruction in all other types of course 
will be documented as far as the available capa- 
city makes this possible. Thus, there will result, 
among other things, possible points of departure 
for later specific evaluation activities. 

Formative evaluation is aimed not only at course 
content and organisation. In accordance with the 
intention of the OS, comiselling, acceptance and 
application procedures, as well as forms of co- 
operative work and self-administration will be 
modified on the basis of empirical data so that 
optimum forms may be found in accordance with 
the OS objectives. 

Proof of transferability of OS products will initially 
be given in constant discussions with representa- 
tives of comparable institutes and publications on 
the situation in curriculum development This pro- 
cess of mutual exchanges can lead to alterations in 
the expectations of the user and modifications of 
the supplied product 

After the first group of students has completed 
the course a further contribution to proving trans- 
ferability of OS products will be made with the 
help of summative evaluation. But proof will also 
be given of the more satisfactory nature of OS 
sdiool and college careers as opposed to traditio- 
nal careers. At first only preliminary work can be 
done on this subject Initial data will be collected 
from all accepted students, data which will be re- 
lated to the OS objectives. On the basis of these 
data, information can be distilled at the end of the 
course about changes among the students with re- 
spect to these objectives. At the present time, how- 
ever, no precise data can be given about the form 
and content of the summative evaluation. 



RUHR TEACHERS' COLLEGE 

Research projects of the Working Group for School Development Research (AFS) 
at Ruhr Teachers' College 

by Professor H.-G. ROLFF, 

Arbeitsstelle f tir Schulentwicklungsforsdiung, 

Dortmund. 



The Structural Plan of the Germtn Educational 
Council and the General Educational Plan of the 
Federal and State Commission on Educational 



Planning (BLK) call for extensive reform measures 
for the present German tripartite sdiool system. 
The age level school (Stufensdiukv is considered a 
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suitable classification system for the future sdiool 
system. For the majority of members of the BLK, 
the objective is the realisation of the integrated 
comprehensive sdiool within the framework of the 
age level school, whereas some of the states are 
still reserving a decision as to type and extent of 
co-operation and integration in the future sdiool 
system. Basically, however, the demand still exists 
that problems or re-organisation be made the ob- 
jecl of scientific researdi and development; only 
in this way can sound solutions to practical pro- 
blems in sdiool reform be formulated. 

This re-organisation problem is the subject of re- 
seardi at the Working Group for Sdiool Develop- 
ment Researdi (AFS). It is being treated in various 
projects, eadi under different aspects. In eadi case, 
the objective is to develop models and planning 
guidelines through interdisciplinary researdi. These 
models and guidelines should help those concerned 
to master pn>blems arising in connection with re- 
organisation at all levels of action. Sudi models 
can only fulfil their purpose, however, when they 
are connected with the development of new curri- 
cula and suitable forms of social organisation in 
sdiools. 

For this reason the AFS begins with a complex 
definition of sdiool development planning (SEP). 
It defines SEP not only as "physical planning", i. e. 
planning of sdiool infrastructures to meet require- 
ments, or rather demographic forecast and corre- 
sponding sdieduling and location of new, supple- 
mentary and re-organised equipment. Rather, the 
AFS wants to conduct its researdi and development 
work with an eye to "substantial" sdiool develop- 
ment, i. e., to co-ordinate it with the development 
of new curricula and corresponding teadiing and 
learning forms. In principle, sudi a complex or ex- 
tensive definition of sdiool development planning 
affects all levels of sdiool administration: the in- 
dividual sdiool, comprehensive sdiool supervisory 
boards, communities and state ministries, training 
and researdi. 

Sdiool development planning can only do justice to 
the interests of those involved and only be realised 
in a democratic manner, however, when it is con- 
ceived as "sdiool-related". For the AFS, sdiool- 
related means that wotk is not done on develop- 
ment of isolated experimental sdiools, but rather 
that transferable models are formulated in close 
co-operation with sdiool and administrative 
practice. Sdiool-related also means that starting 
points of participation must alw*ays be taken into 
account. Nevertheless, it is necessary — if only in 
order not to be overrun by narrow-minded 
individual interests, — to integrate sdiool develop- 
ment planning into the entire social and educational 
framework. Thus it is absolutely necessary to 



strive for systematic connections between indivi- 
dual development models and social diange in 
existing sdiools. 

This definition of complex sdiool development 
planning, here outlined only briefly, will be 
explained more fully in the following sketdi, whidi 
attempts to present the individual components and 
interdependences of sdiool development. 

Researdi and development activities of the AFS 
are divided into four projects, whidi may be 
briefly sketdied as follows: 

Project A: Adaptation planning for sdiool level 
system: "Seminars on Sdiool Develop- 
ment Planning and Development of 
Standardised Procedures" 
Project Head: Dr. K. Klamm 

In the first phase of this project, in progress since 
the summer of 1973, basic problems of method in 
sdiool development planning (SEP) were worked 
on, studied and operationalised for planning of the 
age level sdiool. This work was tested in trcining 
seminars for sdiool administrative experts. Expe- 
rience gathered from them has been doctmiented 
in a publication on SEP methods. 

Independently of the further development and 
especially the evaluation of the initial attempts to 
work out methods and processes in SEP, the work 
of the project also involves problems arising with 
the sdiools in realising an SEP plan after it has 
been set up. 

Problems of implementation after setting up an 
SEP are formulated and dealt with pnmarily in the 
light of the following: 

— Opportunities for participation in the prepara- 
tion, discussion and implementation phase of an 
SEP 

— Combining the SEP with sdiool planning 
(sequence models for realisation of the planned 
age level school) 

— Simulation models for pupil movements and 
evolution of pupil-numbers with alternative 
assumptions on transition and success quotas; 
comparison and refinement of prognosis techni- 
ques. 

Project B: Adaptation planning for sdiool level 
system: "Development of Models for 
Standard Equipment and Sdiool Man- 
agement Systems for Lower and Upper 
Secondary Sdiool" 
Project Head: Prof. H. Fromberger 

This project is meant to study what standard 
equipment systems for age level schools must be 
like to correspond to the modified requirements of 
curriculum and social organisation. 
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The first focus of the project is: 

— development of generalisable and transferable 
evaluation procedures for existing equipment, 
and» 

— preparation of a catalogue of sudi equipment 
systems with the object of determining maxi- 
mum and minimum equipment programmes for 
level schools. 

The second focus of the project is concerned with 
improvements in sdiool management systems in the 
Upper and Lower Secondary sdiool systems, 
frequently larger than present ones; researdi 
subjects here are: 

— to what extent proposals on reform of educa- 
tional administration can be realised; and 

— to what extent management tediniques from 
administration and business can be applied to 
schools. 

In order to answer these questions, experiences in 
existing school centres will be analysed. 

Project C: Adaptation planning ior sdiool level 
system: "Development of Models for 
Regional and Local Advancement 
Planning for Sdiool Centres at Upper 
Secondary Sdiool Level" 
Project Head: Dr. G. Hansen 

Models for sdiool centres at Upper Secondary 
level (S II) for local and regional advancement 
planning are to be developed. Using concrete 
examples of sdiool pilot reforms from regions of 
varied structure, the pre-requisitps for co-operation 
and integration at Upper Secondary School level 
are being studied. 

The following questions are treated in detail: 

— Location of S II centres as an integral part of 
regional development planning in connection 
with social and locational disparities; 

— Definition of optimum operational sizes with 



relation to curricular objectives, social organi- 
sation, etc.; 

— Possible use for existing older buildings; 

— Expanded use of the S II centres through 
extra-curricular public work (adult-education 
activities, youth work, etc.); 

— Organisational and subject requirements of 
co-operation and integration. 

This project is related to pilot projects in North- 
Rhine Westphalia. 

Project D: "Social Organisation of Sdiool Centres 
at Lower Secondary Sdiool Level" 
Project Head: K.-J. Tillmann 

This project considers re-organisational problems 
of existing vertical sdiool forms from the point of 
view of optimum internal structure of the indivi- 
dual sdiool and its social environment. It is from 
this angle that the question is to be studied as to 
what conditions are necessary to permit conversion 
of various existing sdiool forms in a school centre 
(secondary modem sdiool, intermediate sdiool, 
grammar sdiool) into an integrated system. This 
project, as are the others as well, is a combined 
researdi and development project: social processes 
in multilateral sdiool centres are to be studied in 
order to provide guidelines for development of 
integration models. The action proposal to be 
derived from this may be defined on two levels by 

— indications of social environments and internal 
sdiool points of departure whidi have a parti- 
cularly positive or negative effect on integration 
processes; and 

— indications of possible successful measures whidi 
could be taken at the various political levels 
(ministries of culture, sdiool administrations, 
participants) to promote at least medium-range 
integration. 

This project is related to pilot projects in Hessen. 



BADEN-WUERTTEMBERG 

Evaluation of school reform pilot projects in Baden-Wuerttemberg — IBS concept and 
initial experience 

by R. H. WEISS, 

Institut fur Bildungsforsdiung und Studien- 
beratung, Stuttgart. 



General problems 

The Institute for Educational Planning und Infor- 
mation in Stuttgart (IBS) was requested in May 



1970 by the Baden-Wuerttemberg Ministry of 
Culture to prepare a concept of evaluation of 
sdiool reform pilot projects which could be used in 
practice, to co-ordinate all secondary measures 
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and to prepare a central evaluation and analysis of 
the results of the experiments for the entire pilot 
project. A year and a half earlier, educational 
scientists at the Ministry of Culture were occupied 
under research orders with similar or other rele- 
vant questions: 

Prof. G. £igler» with the aid of a team of co- 
workers at Mannheim University, prepared a 
concept of evaluation founded on scientific theory 
whidi has now been published in Series A of the 
Ministry of Culture, Vol. 23. 

Prof. A. Flitner (Tuebingen University) was con- 
cerned with problems of curricular re-orientation 
and the development of a r — curriculum, parti- 
cularly during the planning p^ase of pilot sdiools 
as pan of the "Working Gr ip Pilot Reform 
Sdiools**. 

Optimum "evaluation** should, in our opinion, 
include two essential aspects: 

— evaluation as an empirical efficiency diedc on 
learning objectives, learning content and ^earn- 
ing organisation, and 

— evaluation as counselling for pilot sdiools. 

Since each model needs a certain "starting-up" 
phase, the focus of activities should initially be the 
counselling field. If only for systematic reasons in 
an experimental set-up. Part I will first of all deal 
with the empirical efficiency dieck. 



PART I 

Evaluation as empirical efficiency check 

/. Comparison of IBS concept and Eigler^s heginn-- 
ings 

The "First Concept for Evaluation of Sdiool Reform 
Pilot Projects*' prepared at the IBS in Stuttgart in 

1970 served as a basis for measures of evaluation 
and counselling by the IBS. This concept, whidi 
was limited to pilot sdiools in Weinheim, Freiburg- 
Hasladi, Markdorf and Bodnegg, has now been 
expanded to include other pilot sdiools: Waldkirdi 
High Sdiool, Second Ulm Model, Heidenheim 
Practical High Sdiool and Osterburken All-day 
High School were included in the programme in 

1971 at the order of the Ministry of Culture. 

Prof. Efgler's report, initiated by the Ministry of 
Culture, proceeds from a theoretical assumption 
about the relationship between planning, political 
decision-making and empirical diedcs, as well as 
between organisational form and performance of 
model schools; the socio-economic aspect of evalua- 
tion, as well as the aspects of "optimum achieve- 
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ment advancement, social equality and social 
integration" are particular points of emphasis. The 
IBS concept, on the other hand, proceeds more- 
from an .operationalisation of the aims formulated 
in the individual planning groups and commissions 
at pilot sdiools, which lends a more individual and 
socio-psydiological investigation procedures in 
assessing abilities, sdiool adiievement, personality 
dimensions and social fields. Abilities and limita- 
tions of "efficiency comparisons" between various 
forms of pilot sdiools and sdiool systems are de- 
scribel in the IBS paper, in whidi, as opposed to the 
Eigler concept, no comparison is denied between 
the conventional sdiool sy^item and new pilot sdiool 
systems when the absolute pre-requisite of exact 
definition and operational representation of general 
learning objectives is given at the end of Secondary 
Sdiool Level. The significance of the question of 
"Economic feasibility*', to whidi Eigler devotes 
mudi attention, is not impaired. 



11. Expanded study hypothesis 
1. Pre-requisites for experiment 

Under the assumption that a specific new form of 
sdiool organisation is already the only alternative 
to existing regular sdiools, one could limit evalua- 
tion solely to this sdiool form, i.e., perceive only 
system-related or sdiool-related tasks (evaluation 
aspect). This pre-requisite, however, is not given in 
the case of the Baden-Wuerttemberg project pro- 
gramme, as it has a multi-level design. Thus an 
evaluating scientist trying to be objective is forced 
in conceiving his researdi design to use not only 
sdiool-related or system-related hypotheses, but to 
study the .given organisational forms of pilot 
sdiools with respect to their pedagogical, psydio- 
logical, socio-cultural and economic effects compa- 
ratively as well. A "scientist" with a one-sided 
political and ideological orientation would relate 
to the one type of sdiool reform whidi best con- 
forms to his idea of a sdiool. This is true of both 
main brandies. The head of a reform, for example, 
who is oriented to the traditional Humboldt under- 
standing of education and is, in addition, interested 
only in high-sdiool "elite formation", would 
concentrate his studies on testing structural 
dianges in the hi^h-sdiool field. It would be just 
as one-sided, however, for politically mterested 
scientists to accept the general sdiool as a matter 
of course already, categorically reject any other 
sdiool form and thus discriminate against compara- 
tive school studies. 

Methodological objections would be more to the 
point here; the less agreement found on common 
learning objectives and the more often objectives 



are modified in the course of one experiment » the 
less decisive the results of a comparative study will 
be. Socio-integrative behaviour as an objective and 
elite consciousness are, in my opinion, mutually 
exclusive; they are incompatible. One of the more 
important conditions for. comparative sdiool re- 
seardi is thus agreement on and precise definition 
of the general learning objectives (e.g., achieve- 
ment objectives in specific subjects, social integra- 
tion, ability to criticise, etc). Another condition is 
that any school form under discussion requires 
counselling by trained experts. This counselling 
must not be limited to pupils and parents, but must 
also include teadiers (through ♦eadier training) and 
thus the classroom as well (teadiing aids, curricu- 
lum development). 



2. Operational definition of objectives 

Basically, it must be mentioned that statements 
about the "value" of various sdiool forms are just 
as limited and relative as statements about the 
"value" of a human being. The conditions of a 
verification of falsification hypothesis safeguarded 
by statistical probabilities depend on the *'value 
references'/ of the investigator. If a diagnostician 
analyses only intellectual achievement in a single 
psychological study, this evaluation is certainly 
just as one-sided as a chec^ on a school experiment 
which measures only the increase in subject 
knowledge. This type of study would be legitimate 
only if the achievement principle is the first in the 
hierarchy of values. 

Now, however, objectives have been developed in 
the planning studies on all pilot schools to be 
evaluated; and these objectives extended far beyond 
the achievement principle. 

A quantification of these objectives (in the form 
of a comparative chedk) is m^rc severely limited, 
because as empirical studies have shown, achieve- 
ment diagnoses can be made up to 90 Vo accuracy, 
but diagnoses of various personality fields (charac- 
teriological dimensions of a non-cognitivecharacter), 
however, with only 50 Vo accuracy. 

Summarised, the objectives of the pilot schools are 
concerned mainly with improvements (increases) 
in the following fields: 

— improvement in cognitive function (e.g., increase 
in intellectual achievement or increase in learn- 
ing ability; building complex linguistic struc- 
tures; 

— personality improvement (e.g., conflict reduction, 
motivation increase, overcoming fears); 



— Socio-integrative improvement (e.g. overcoming 
social tension, social communication abilities). 

The scope of problems for scientific evaluation of 
the efficiency of school reform pilot projects with 
such comprehensive objectives can thus be formu- 
lated as follows: 

a) How can I measure whether the objectives have 
been attained at a pilot school (school-related 
camparison)? 

b) How can I measure whether these objectives 
were attained more efficiently ir one pilot 
school than in «inother comparable pilot school 
(inter-model school comparison)? 

c) How can I measure whether these objectives 
were attained more satisfactorily than in the 
conventional school system (inter-system com- 
parison)? 



PART II 

Evaluation as counselling for school reform pilot 
projects 

Counselling of pilot schools is an integral part of 
evaluation. 

The following fields are part of scientific coun- 
selling of pilot schools by the IBS: 

2. Pupil, teacher, and parent counselling and tea<^' 
ing aid 

Work in this area belongs to the fields of educa- 
tional counselling centres and psychologists at pilot 
schools. The work of a Master of Psychology at a 
pilot school can be summarised as follows: 

— Pupil counselling: 

Diagnosis, counselling and caring for develop- 
ment-disturbed children with learning and 
conflict problems; advancement diagnosis and 
counselling (e.g., for legasthenics). 

— Parent counselling: 

Counselling in case of childrens' difficulties at 
school and disturbed behavioural patterns; 
information on possible school careers and 
occupations for pupils, as well as on details of 
pilot school evaluation. 

— Evaluation and counselling: 

Co-operation in evaluative studies (feeding areas 
initial and progressive studies); participation in 
general teacher conferences and specialised 
teacher conferences on curriculum planning and 
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revision; counselling on improvement of achieve- 
ment assessment in problems of teaching 
differentiation and other pedagogical and 
didactic measures. 



2. Counselling and co-operation in curriculum 
development at school reform pilot projects 

"Curriculum" is defined here in the broadest sense, 
including questions of achievement assessment and 
differentiation. 

The curricular development work of the pilot 
sdiools should be organised on the principle of 
"scientific suburbs**, meaning that one authority, 
e.g., a studies seminar, a teachers' college or a 
university institute, would be in diarge of curri- 
cular work in one subject and that the pilot sdiools 
would be given the teaching units, tests etc. 
developed there. The work of the commissions at 
the individual pilot schools would be co-ordinated 
by the scientific suburb. In this system, the IBS 
would perform the function of documentation and 
transmission of this work. 



GLOSSARY 

Comprehensive school 

"Multilateral" comprehensive sdiool 
'^Integrated" comprehensive school 

Grammar sdiool 

Intermediate sdiool 
(in the narrow sense) 

Secondary modem sdiool 

6th form college 

Comprehensive 6th form college 
(academic and vocational) 

All-day sdiool 



Norway 



Educational innovation in Norway 

The 1970s ai^ from many points of view an 
interesting period in Norwegian sdiool history. The 
reform of the primary school (the 9-year compul- 
sory comprehensive sdiool) has just ended with the 
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This organisational principle has not been rejected 
completely but nonetheless, strongly modified in 
practice. The commissions operate independently 
of one another for the most part, which is due on 
the one hand to practical difficulties in co-operation 
(lade of time, travel expenses), and on the other to 
special conditions at individual sdiools whidi would 
not permit immediate adoption of curricula deve- 
loped for other sdiools. Since the interest of the 
individual commissions in curricula developed at 
other locations has now grown considerably, how- 
ever, (individual teadiers overloaded), one authority 
must organise documentation of all curricular work 
and its distribution. This function is assumed by the 
IBS. Useful work is only possible, however, if the 
pilot sdiools regularly send their working papers 
to the IBS. With some sdiools, this process is 
already being carried out to perfection, with others 
there are still some difficulties. 

In addition to this, however, the IBS is already 
able to co-operate as counsellor in the individual 
commissions and also develop its own curricular 
sections and will be able to do so even more in the 
future. 



Gesamtsdiule 

Kooperalive Gesamtsdiule 
Integrierte Gesamtsdiule 

Gymnasium 

Realsdiule 

Hauptsdiule 

Oberstufenkolleg 

Kollegstufe 

Ganztagsdiule 



by Mr. O. LINDAL, 

Lecturer, College of Education, Stavanger. 



presentation of the new Curriculum Plan (1974), 
but many problems and questions are still unsolved. 
The upper secondary school reform has been under 
preparation for many years, and this year (1974) 
a new law introducing an all-round sdiool system 
for further education (16-19) has just been pre- 
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sented, together with a new Curriculum Plan for 
this type of school. A new law, new regulations 
aiid a new Curriculum Plan for teacher training 
will be introduced next year (1975). The first 
experimental 3-year period for the district colleges 
has just finished. 



Decentralisation of decision-making authority 

Trends towards greater autonomy at lower levels 
in the educational system are present to some 
degree. In higher education a gradual transfer of 
the decision^making authority has taken place, 
especially where the distribution of resources is 
concerned, from the state to the large institutions. 
This is most marked at the new district colleges, 
where the schools' own managing bodies have 
comprehensive powers over the allocation of the 
resources at their disposal. 

Like the universities, the district colleges also have 
great freedom in deciding the content and planning 
of the instruction are concerned and in their 
methods of evaluating the students. 

At lower sdiool levels there has oeen a transfer of 
the decision-making authority to the counties for 
the upper secondary school and to the communities 
in connection with the 9-year compulsory compre- 
hensive schools. Work is also being done on a 
reorganisation of the subsidies system for the com- 
pulsory sdiool, whidi will render superfluous the 
detailed regulations aimed at limiting the use of 
teadier resources. In this way important decisions 
on internal resource allocation can be delegated to 
the local authorities. The distribution of the 
decision-making authority between community 
bodies and the individual schools has not yet been 
clarified in this context, however. 

Vxe students' range of dioices in the courses of 
idy and subject combinations has increased at all 
.1 the secondary sdiool levels, and the Curriculum 
Plan of 1974 for the 9-year comprehensive sdiool 
has more of the diaracteristics of an indicative plan 
than was previously true, at the same time as it 
allows somewhat greater freedom in the distribu- 
tion of instruction hours among the various sub- 
jects. Reduced emphasis on examinations in the 
9-year sdiool should also result in a wider range of 
dioices where the instruction plan is concerned. 

Among the vocational schools in particular there 
are individual examples of close contact between 
the sdiool and the local community, first and fore- 
most with local business. The new act on 
secondary sdiool allows for stronger local repre- 
sentation more strongly into the management of the 
sdiool's activities. The district colleges have, in 



part, been very successful in achieving an active 
co-operation within their regions. In the 9-year 
comprehensive schools the establishment of new 
co-operative bodies offers possibilities for expanded 
contact with parents. 

Until now the developments in this country towards 
a closer integration of the school units with the 
local environment have been very modest. These 
are, in general, cautious efforts to expand the 
contact surface with the local milieu, but these 
efforts have not really aimed at making any far- 
reaching changes in the decision-making process 
itself. 

In the Norwegian educational system too, in recent 
years, a number of features in the decision-making 
process have emerged that deviate from the 
patterns we recognised earlier. Those trends that 
are being expressed can, however, in many cases 
be considered as conflicting trends. Increased 
interest in global problems goes hand in hand with 
greater concern with the problems of the local 
community. Attempts at subject integration are 
being made while at the same time the new, 
programmed instruction plans are based strictly on 
the "inner logic** of the traditional disciplines. 
Greater emphasis is being placed on the relevance 
to the students of the instruction, while at the same 
time the instruction is being programmed on the 
basis of given objectives, etc. 

Anyone seeking a clear development trend in this 
area will find it difficult to find any answer in 
today's situation in the Norwegian educational 
system. Perhaps it will gradually prove that some 
of' the seemingly conflicting trends in today's 
situation are not necessarily in conflict with each 
other at all. 



The background for the reforms 

The social and cultural patterns of Norway are to 
a large extent shaped by the geographical condi- 
tions of the country and cannot be fully understood 
unless seen against this badcground. Norway has a 
population of only 4 million, but in terms of area 
it is a fairly large country by European standards: 
more than 125,000 square miles (about the same 
size as the United Kingdom). Thus Norway is easily 
the most sparsely populated country in Europe 
with about 30 inhabitants per square mile. The 
isolation this creates is increased by the topography 
of the country and by the climatic conditions in 
winter. It follows from these facts that the country 
consists of small and in many cases rather isolated 
administrative units. Educational administrative 
authorities have to cope with a large number of 
small school districts and small sdiools. 
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Urban and rural sdiools in Norway have always 
operated tinder different conditions. Urban and 
rural primar^ schools therefore used to follow 
different curricula, though the subjects were the 
same and the pupils were supposed to readi the 
same standard. Up to 1959 there two separate laws 
on primary education, on for schools in towns and 
one for sdiools in the country. The law of 1959 
strengthened the rural school and formed the basis 
for further equalisation. 

In secondary sdiools as well the differences be- 
tween urban areas and sparsely populated districts 
is strongly felt. In Oslo, for instance, almost every 
student goes to a secondary school when he has 
finished his primary education, but this is far from 
the case in, for instance, Finmark. 

One of the most striking features of the develop- 
ment of Norwegian public education is the growing 
wish to give all diildren an equal opportunity for 
education. 

Three sets of objectives, with differing content, can 
be brought out in this context: 

— Equality of access to education, without formal 
discrimination in relation to any group. 

— Equality in the distribution of educational re- 
sources among different groups, with compen- 
sation for those groups that have a poor 
starting point. 

— Equality in performances within the school 
system among different groups. 

This development is closely related to general 
social, cultural and economic trends, and reflects 
the growth of democratic forces since the middle 
of the 19th century. 

Tl ^ educational reforms in Norway ought not to be 
st-iied in isolation; they must be compared with 
similar reforms in the other Nordic countries and 
other European countries. These societies are going 
through a similar development as Norway: techni- 
cal innovations and increased industrialisation 
whidi create a complicated society demanding new 
knowledge and new abilities from its citizens. As 
a consequence, the sdiool system must be in con- 
stant development in order to prepare the students 
for a dianging society. 

The education reform programme 

The point of departure for the Norwegian educa- 
tion reform programme is to be found in the 
so-called Joint Programme of the Norwegian 
political parties announced in 1945.. This includes 
the following statement on educational develop- 
ment: 
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"The entire school system must be co-ordinated so 
that the transition through its individual links, 
from the elementary to the highest educational 
level, might evolve smoothly from one to another, 
and this should apply equally to practical as well 
as literary types of sdiooling." 

On the basis of this programme the Norwegian 
Labour Party, as the party in office for the next 
two years, assumed the task of putting into effect 
this programme for the democratisation of the edu-^ 
cational system. 

The reform of the educational system demanded 
an experimentation in sdiool hitherto unknown. 
The eJcperimental activity was included in a 
number of measures designed to improve the 
sdiools in general. These had two objects: the im- 
provement of the "old" sdiool and also the out- 
lining of the new school system for the future.. 



Research and experiments 

In 1954 it was resolved to establish a body (the 
National Council for Innovation in Education — 
NCIE) that would be of help to the Ministry "with 
advice, initiative and supervision on experimental 
activities in the sdiool". In this context it was 
stated in the Storting (the Parliament): "By em- 
ploying empirical science in order to try out new 
sdiool forms and educational methods, the educa- 
tional system will have guided the work of reform 
in this important section of the social system into 
modem forms. These methods are being used today 
in every other field in our society, and they ought 
to be used as a matter of course in education. By 
utilising running research projects in schools, one 
should be able, as far as both organisation and 
teadiing are concerned, to manage to keep educa- 
tion abreast of current development whidi is 
taking place in our society. It must be admitted 
that our sdiool reforms have not always been made 
early enough to satisfy those demands which 
society makes on the school at any given time. This 
is due to a great degree to the long study and 
reporting process which has been followed up 
until now," 



The Innovation Act gave the Ministry the following 
powers and responsibilities: 

"His Majesty's consent having been obtained, to 
disregard) for purposes concerned with experimen- 
tation^ the regulations embodied in the appropriate 
sdiool lawS) whenever sudi experimentation may 
be deemed well founded from the educational point 
of view and in the interest of the sdiool." 
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"In order to obtain authority for experimental 
activity, the NCIE shall submit to the Ministry of 
Education plans for experimental instruction or 
shall pronounce on plans put forward by individual 
schools and others. This shall also apply to objec- 
tives and the qualifications such instruction may 
confer." 

"Reports shall be made to Parliament every year 
on the subject of experimental activities initiated 
under this law." 

This act might be looked upon as an interesting 
strategy in coping with a traditional educational 
bureaucracy dependent on laws and regulations. 
One important aspect of this law is that it gives re- 
search and development a change to influence 
entirely new educational organisations. Another 
aspects is its relationship to the political and 
administrative decision-making bodies. The Minis- 
try has to approve the NCIE's budget and the 
appointments, and in policy questions the Ministry 
can use the NCIE deliberately as its implementa- 
tion arm for policy recommendations. 



NCIE activities 

The National Council for Innovation in Education 
(NCIE) began its activities in October 1954. 

Although it was stated that experiments were 
desirable in all types of school, it was the 9-year 
comprehensive school that became the main field 
of activity, up until 1970. Since 1970 the main 
emphasis in the Council's work has been placed on 
experiments with and the setting up of the upper 
secondary school — the integration of the earlier 
Gymnasium and vocational education. 

Most of the teacher training colleges have over the 
last 10-year period taken part in a more Of less 
comprehensive experimental activity. In 1972 
approximately 70 % of the total offers of training 
at the colleges were made according to the Act of 
Innovation. 

For 20 years now the NCIE has been the main 
agency for promotion of large-scale educational 
innovation in Norway. It is closely linked to the 
central educational administration and to the 
hierarchy of the decision-making structure. The 
task given to the NCIE extends to almost all types 
of schools in Norway and to the entire country. 



Types of innovations 

Types of innovations in Norway can be classified 
in three groups: 



Major reforms 

One example of this type is the introduction and 
setting up of the 9-year comprehensive school. 
Basically, the intention was to introduce a new 
structure and curriculum and to plan to establish 
this new educational provision throughtout the 
country, school district by school district. The re- 
form was based on proposals from several com- 
missions and political decisions by the Ministry 
and Parliament. 

The main role of the NCIE in this phase of the 
reform work has been to find answers to specific 
problems defined by the Ministry or the Parlia- 
ment. The first phase of an educational reform of 
this kind is to establish the new structure, organi- 
sation and administration. 

Curriculum development 

The second phase of the above-mentioned educa- 
tion reform is specific curriculum development 
projects. Most of them are in our case aimed at 
independent learning/individualised instruction. 
The projects have had a more strict planning- 
research-and-development cycle. 
These types of innovations are mainly concerned 
with what is assumed to be common needs for all 
the schools in the country. 

Local projects 

The curriculum development projects, based on the 
R&D approach, could reach many schools since 
they were based on the notion of ''packaging" 
expert knowledge and services, which could then 
be disseminated throughout the country. Yet, 
feasible as it appeared, this strategy has not proved 
to meet the specific needs of the great variety of 
schools in the country. A new type of innovation 
therefore has been developed over the ^ast years. 
One of its most important objectives iias been to 
meet the needs of the individual school. Another 
objective is the investigation of the possibility of 
giving schools and local commutaties more freedom 
and initiative. This is, in itself, a ir.aior departure 
from a centralised strategy for innovation. 
The so-called "autonomy project", "open schools" 
and the establishment of regional and local deve- 
lopment centres can be looked upon as examples 
of a new strategy for innovation; decentralisation 
of decision-making and resources. 

Zdenti/ication of educational needs 
The identification of educational needs in Norway 
is traditionally done through Royal Commissions 
which make recommendations to the Ministry and 
Parliament. The NCIE tries to operationalise these 
objectives into feasible school practice. 
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The establishment of priorities through commis- 
sions is seldom guided by research. The process 
within NCIE, however, has over the last 5-year 
period been more and more related to research 
and development projects which are carried out in 
order to clarify the functioning of the system as it 
exists. Evaluation is in some cases done outside the 
NCIE on contract. 

Some university research projects have also proved 
to be of great importance for the ongoing discus- 
sion on educational needs. An example is the 
identification of the needs for new experiments in 
the 9-year comprehensive school: the 1960s were 
characterised by a desire to develop and strengthen 
the 9-year comprehensive school. Extensive experi- 
ments were initiated in order to develop suitable 
plans and programmes for the school's structure, 
curriculum, methods, differentiation, and evalua- 
tion. 

Already early in this period it became evident that 
the 9-year comprehensive school was facing far 
greater problems than the 7-year primary school 
ever had. An important problem during the period 
of experimentation was how to create a school 
which all pupils — regardless of abilities and 
ambitions — can find interesting and rewarding, a 
school which can give all children opportunities to 
develop their special talents and potentials. 
It is, of course, an extremely difficult task to 
develop such a school. All the problems which this 
task raises, are far from solved. But much think- 
ing and work is being done in order to develop a 
better school for all the children. 
In this connection it might be relevant to point to 
some of the most important results of the research 
study based on interviews made by Professor 
Sandven, University of Oslo, in 1966 and 1970. 
Professor Sandven's aim with this study was to 
find out what the pupils themselves thought about 
their school situation, whether or not they liked to 
go to school. 

— About one-third of the pupils had ambivalent 
attitudes towards their school situation. Seven per 
cent of these pupils said that they were seldom 
happy at school. That means that 2.3 of the 
total number of pupils are seldom at ease while 
at school. 

— The pupils from the most demanding "course 
streams'' (') were happier at sdiool than pupils 



(1) The "course streaming system" meant that pupils 
in the 8th and 9th grades were taught with their 
classes, except for the subjects Norwegian, English, 
German and mathematics. These subjects were di- 
vided into different levels of performance, "course 
streams". In Norwegian, English and mathematics 
there were three sudi levels. In German — whidi 
is an elective subject — there were two levels. 



from less demanding "course streams". According 
to Professor Sandven, this may suggest that our 
comprehensive school is a good school for those 
pupils who are interest ad in theoretical subjects, 
but is less suitable for those who are more 
Interested in practical subjects. 

— When asked about how :hey found their actual 
school work, a still greater number of pupils 
answered that they found it uninteresting. As many 
as 43 of the pupils said that they disliked their 
school work. There were marked differences be- 
tween pupils from various course streams. 

— Thirteen per cent of the pupils from the ninth 
grade (in 1966) answered that leaving school before 
the final examination might be a possible solution 
to their problems. Among the pupils from the less 
demanding course streams as many as 20 Vo con- 
sidered leaving school. But only 2.5 of the 
pupils from the more demanding course streams 
had considered this possibility. 

— In 1970, an increased number of pupils in the 
ninth grade wanted to leave school at once: the 
percentage was now as iiigh as 17 (26 in the less 
demanding course streams and 5 in the more 
demanding course streams). 

Teachers also participated in this stud v. 

According to the teachers, the main reasons for the 
problems in the 7th, 8th and 9th grades of the 
comprehensive school are the following: 

— The school does not represent an answer to the 
pupils' actual needs. Their need for emotional se- 
curity and stimulation is not catered for. The school 
of today stresses the pupils' intellectual develop- 
ment far too much. 

— The teachers "lecture" too much while at school, 
therefore the pupils get few opportunities for 
individual and creative activity. 

— The schools ought to have a greater degree of 
autonomy. Schools are often too big, and there are 
often too many pupils in each class. The pupils, 
especially those who are less interested in theore- 
tical subjects, should get more information about 
their future opportunities, both about education 
and working conditions in a modern society.. 
"Vocational guidance" is a very important subject 
in the school, and it ought to be strengthened. 

— Parents are - in many cases - too ambitious on 
behalf of their children. This may lead to a 
situation where the children feel that they are 
forced to do better than they are really able to, in 
order to live up to their parents' expectations. 
Parents - as well as the school - have a tendency 
to value theoretical capacity higher than practical 
capacity. 
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Professor Sandven hss given some comments upon 
the results of his stucy. He claims that the follow* 
ing conclusions could be drawn: 

— The aim of education should be to develop all 
the child's possibilitt<?s and talents, not only his 
intellectual capacity. 

— The educators* attitude towards their task ought 
to be dianged. They ought to be more interested 
in the child as a growing personality than in the 
subject they teach. Curriculum plans ought to be 
more flexible. The individual school or community 
must be free to develop their own programmes. 

— One important aim in education must be 
stressed: the pupils must regain their self-confi- 
dence which they - as small children - had in 
their own capacity to learn. They must be moti- 
vated to develop their own personality. 

— The watertight division between "theoretical 
subjects" and "practical subjects" should be torn 
down both in school and in work. 

— All pupils must develop the fueling of having 
succeeded at school. Therefore, the school must not 
make demands that only a small number of pupils 
are able to fulfil. 

These conclusions reflect much of the thinking 
behind the third type of innovation mentioned 
above. 



Planning of innovation 

Most of the planning of innovation in Norway has 
until recently been done in the NCIE, which has 
had considerable authority in this respect. The 
planning has been based on wide contacts and 
co-operation with local authorities and teachers. 

In its first years of operation the NCIE to a large 
extent decided the regulations to be applied for the 
implementation of a reform. During the 1960s one 
can observe a change in the role of the NCIE. In 
the early 1970s the planning has taken the form 
0/ consultancy and advice to the different partici- 
pating bodies in an innovation. The responsibility 
for innovation in education is more and more 
divided between several agencies. Each of the new 
laws mentioned above (para. 1) contain one section 
about innovation and further development in the 
schools. No'- e teachers, the individual schools, 
the local sch boards and the different councils 
under the M. ''y can all take the initiative, plan 
and carry ou .ovation within the broad frame- 
work of the nevv laws and regulations. Only the 
innovations that go beyond these new extended 
boundaries have to be carried out according to the 



Act of Innovation, under the guidance of the NCIE, 
and in co-operation with any other council involved. 

The colleges of education, for example, now have 
to fulfil a twofold task in innovation: 

— Innovation aimed at the teacher training itself. 

— Innovation in the different types of schools they 
train teachers for. 



Model for development in structural reforms 

The model for development in structural reforms 
has not followed a strict planning^research-develop" 
meni cycle. 

The model is much more development-oriented. 
Most of the development work is done in local 
communities or in individual schools, based on 
close co-operation with the participating schools 
and local communities. 

Evaluation in a more strict sense was not very 
much applied in the 9-year school reform except 
for evaluation of pupil performance (examinations 
and standardised tests). 

Moving on to the secondary school reform research 
workers have been following the reform all the 
time, and there is a relatively rich crop of reports 
and research material on different aspects of the 
reform. It is i?\v to say, however, that revisions 
and new decisions are taken after various forms of 
assessment. 

Informal and formal discussions together with the 
more research-oriented follow-up activities are the 
basis for evaluation of the secondary school reform 
as well as in the reform of teacher training. 

In the curriculum development projects, however, 
there is a process which to a large extent follows 
the planning-research-development model: after 
objectives have been carefully outlined and dis- 
cussed with different interest groups, a proposal 
is made about a curriculum development project. 
A project group, or a "reference group" together 
with a project director in the couricirs staff, works 
out prototype curriculum units and tries them out 
in a few schools. An evaluation and revisions cycle 
is based on the first or first two years of experi- 
ments, and a second version of the material is 
prepared and tried out in a larger number of 
schools with teachers who have not helped develop 
the material themselves. Evaluation is done by 
research workers from the NCIE staff or a re- 
search institute together with the project director 
and the project group. Based on the evaluation, a 
revision is made, and sometimes a third and large- 
scale try-out is organised. The summative evalua- 
tion also includes recommendations to the Ministry 
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about the implementation of the new curriculum 
and material in all schools in the country. Imple- 
r4.entation is the responsibility of agencies other 
than NCIE. 

An exemple: "The MotheT-Tongue Project" 

The background for this project is to be found in 
the desire to individualise teaching within the 
ffdmework of the class. The main goal of the 
project is to organise the teaching of Norwegian in 
the 7th, 8th and 3th grades of the 9-year school in 
non-streamed classes. Developing properly diffe- 
rentiated teaching material is another important 
aspect of the project. The project is supervised by 
a committee appointed by the Council, and the 
daily leader of the project is a consultant in the 
Council. 

The project started very modestly in 1964 with 
only one school. At the present stage (1974) 271 
schools all over the country take part in the project. 
The project goes through tv/o phases: in the pri- 
mary stage preliminary experimentation in a few 
schools is carried out. The project is tried out, 
revised and tried again. In the expanded phase the 
project is extended to a larger number of schools, 
still under follow-up and supervision. Researchers 
outside the Council — at the University of Trond- 
heim — study both social aspects and the profes- 
sional value of the project. The expanded 
experiment continues for two years in each grade. 



In the present project the time-table is as follows: 





1971/72 


1972/73 


7th grade 


expand. 


expand. 


8th grade 


primary 


expand. 


9th grade 


primary 


primary 




1973/74 


1974/75 


7th grade 


implementation 




8th grade 


expand. 


implementation 


9th grade 


expand. 


expand. 



As soon as the expanded experiment has been 
evaluated, the teaching material that has been tried 
out will be finally revised and then published 
commercially in co-operation with one of the Oslo 
publishing houses (implementation). 



One main problem in connection with this project 
has been to build up the material in such a way 
that it has something meaningful to offer to all 
categories of students. To solve this problem all 
texts are built up around common introductions to 
each chapter to be taken in full class. All questions 
and exercises that are connected with the intro- 
ductions are differentiated. Different methods may 



be used in working with these exercises. Some of 
the problems may be solved by groups, some of 
them may be solved individually, and some of them 
are solved by the whole cla.ss together. 

Through this ''oject valuable experience is gained 
in co-operatiuit between private publishing houses 
and the school authorities. In some cases these two 
parties may have somewhat differing interests, but 
ways of communicating and reaching agreement 
have been found, and the work has been carried 
through with very little friction. 

The aim of the project was to keep the class unit 
together. Nevertheless, until now the pupils have 
had to choose ** levels" in the spring term in the 
9th grade. For the examination the pupils are 
marked on different scales according to the '"level" 
they have chosen. This is of course very unfortu- 
nate, but the new curriculum plan (1974) is putting 
an end to this. 

The material produced and presented in this pro- 
ject has been received by teachers as well as by 
pupils, and representatives from these groups have 
been involved in material development and evalua- 
tion of the project at all stages in the process. The 
project as a whole has met with very little 
resistance. The smaller schools especially have 
welcomed the opportunity to keep the class unit 
together without having to organise different 
groups for each level. The mother tongue is a 
subject where communication, written of oral, is 
the main aspect, therefore, both teachers and pupils 
seem to find it natural that the whole class is 
studying this subject together. 

The new curriculum plan will be introduced in the 
comprehensive school this autumn for all classes, 
except for the 9th grade. Streaming by levels is 
abolished by this plan, and common evaluation is 
introduced in all subjects. The material produced 
and the experience gained through the mother 
tongue project will probably be of great value in 
this connection. 

An example of the Local projects: the so-called 
^'autonomy project" in Sandnes 

The project is in operation with assistance from the 
Work Research Institutes, has gained its inspiration 
from the experiments in co-operation in working 
life. An investigation is being carried out here into 
the sdiool's technical and social organisation, and 
into the limiting conditions under which it is oper- 
ated, with the effects which these have on influen- 
ce, personal development and learning. 

The experiments have not been started with any 
definite idea of what will be a "correct" solution 




to the problems; changes must follow as a result 
of the school's own work on these questions. But it 
is obvious that the relationship between students 
and teadiers and also the traditional methods in 
the division of subjects must be taken up for eva- 
luation. 

The function o/ research 

In each of the three groups of innovation we have 
discussed in this paper (Major reforms, Curriculum 
development, and Local projects) research is play- 
ing a somewhat different role. The research func- 
tion, however, is a growing concern in innovation 
in Norway. Nearly all major projects, run by NCIE 
or any other agency, are now planned in co- 
operation with researdi workers. One of the major 
problems, however, has been that a decision- 



oriented research process is difficult to establish in 
reforms of a complex nature like the introduction 
of comprehensive sdiools. 

Another problem has been that the money for most 
of the reform and development work has come 
from a different source than the money for the 
research work related to the same projects. This 
situation also reflects the fact that research work 
until recently in many ways has been looked upon 
as something different from reform, experimental 
and development work. This is now changing. Re- 
seardi work is viewed more and more as an 
integral part of all types of innovation, where the 
research worker has more of an assistant partici- 
pant role with observation and interpretation as 
specific tasks in the ongoing process. 
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